diff --git a/.gitattributes b/.gitattributes
index a6344aac8c09253b3b630fb776ae94478aa0275b..c4cb545ce9c9bbbff6607aa2f92b4a57a0851402 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.json filter=lfs diff=lfs merge=lfs -text
diff --git a/bn/baseline/data_15000_1000/README.md b/bn/baseline/data_15000_1000/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..8ec846ec473678d77b1d179353d91189ad17d0ae
--- /dev/null
+++ b/bn/baseline/data_15000_1000/README.md
@@ -0,0 +1,70 @@
+---
+license: llama3.1
+base_model: meta-llama/Llama-3.1-8B-Instruct
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+library_name: peft
+model-index:
+- name: data_15000_1000
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# data_15000_1000
+
+This model is a fine-tuned version of [meta-llama/Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.2074
+- Accuracy: 0.2232
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0005
+- train_batch_size: 25
+- eval_batch_size: 25
+- seed: 1
+- distributed_type: multi-GPU
+- num_devices: 4
+- total_train_batch_size: 100
+- total_eval_batch_size: 100
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: inverse_sqrt
+- lr_scheduler_warmup_ratio: 0.03
+- num_epochs: 5.0
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Accuracy |
+|:-------------:|:------:|:----:|:---------------:|:--------:|
+| No log        | 0      | 0    | 0.3140          | 0.2161   |
+| 0.2032        | 1.3333 | 200  | 0.2120          | 0.2228   |
+| 0.167         | 2.6667 | 400  | 0.2074          | 0.2232   |
+| 0.1357        | 4.0    | 600  | 0.2110          | 0.2233   |
+
+
+### Framework versions
+
+- PEFT 0.15.2
+- Transformers 4.44.0.dev0
+- Pytorch 2.7.1+cu126
+- Datasets 3.6.0
+- Tokenizers 0.19.1
\ No newline at end of file
diff --git a/bn/baseline/data_15000_1000/adapter_config.json b/bn/baseline/data_15000_1000/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..5444ff418752eb22d4d0b56630b185f30328a13c
--- /dev/null
+++ b/bn/baseline/data_15000_1000/adapter_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eaae1e886a57f5c5700fa5cc45e2d7e6aa73ffe52e8820fadf55c5c5cce179ec
+size 863
diff --git a/bn/baseline/data_15000_1000/adapter_model.safetensors b/bn/baseline/data_15000_1000/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..14aced81d1a0ba6975695076582ea155ae3d4f55
--- /dev/null
+++ b/bn/baseline/data_15000_1000/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5b4a528a50374541cfd7eda52cca8c5f350f4ca60788c7468e88e6fbcb7769e
+size 42002584
diff --git a/bn/baseline/data_15000_1000/adapter_model/README.md b/bn/baseline/data_15000_1000/adapter_model/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0d31128190920e45b61115944d16e773c2ec94c3
--- /dev/null
+++ b/bn/baseline/data_15000_1000/adapter_model/README.md
@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/bn/baseline/data_15000_1000/adapter_model/adapter_config.json b/bn/baseline/data_15000_1000/adapter_model/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..5444ff418752eb22d4d0b56630b185f30328a13c
--- /dev/null
+++ b/bn/baseline/data_15000_1000/adapter_model/adapter_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eaae1e886a57f5c5700fa5cc45e2d7e6aa73ffe52e8820fadf55c5c5cce179ec
+size 863
diff --git a/bn/baseline/data_15000_1000/adapter_model/adapter_model.safetensors b/bn/baseline/data_15000_1000/adapter_model/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..14aced81d1a0ba6975695076582ea155ae3d4f55
--- /dev/null
+++ b/bn/baseline/data_15000_1000/adapter_model/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5b4a528a50374541cfd7eda52cca8c5f350f4ca60788c7468e88e6fbcb7769e
+size 42002584
diff --git a/bn/baseline/data_15000_1000/all_results.json b/bn/baseline/data_15000_1000/all_results.json
new file mode 100644
index 0000000000000000000000000000000000000000..748858550e61e1a0258c92ff2bbd15a5fbeda879
--- /dev/null
+++ b/bn/baseline/data_15000_1000/all_results.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5c8f781c0b6e53499b9f0284fb6e4b201035c8f496a86fc117e952aef4a69ee
+size 485
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/README.md b/bn/baseline/data_15000_1000/checkpoint-400/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0d31128190920e45b61115944d16e773c2ec94c3
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/README.md
@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/adapter_config.json b/bn/baseline/data_15000_1000/checkpoint-400/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..5444ff418752eb22d4d0b56630b185f30328a13c
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/adapter_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eaae1e886a57f5c5700fa5cc45e2d7e6aa73ffe52e8820fadf55c5c5cce179ec
+size 863
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/adapter_model.safetensors b/bn/baseline/data_15000_1000/checkpoint-400/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..14aced81d1a0ba6975695076582ea155ae3d4f55
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5b4a528a50374541cfd7eda52cca8c5f350f4ca60788c7468e88e6fbcb7769e
+size 42002584
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/adapter_model/README.md b/bn/baseline/data_15000_1000/checkpoint-400/adapter_model/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0d31128190920e45b61115944d16e773c2ec94c3
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/adapter_model/README.md
@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/adapter_model/adapter_config.json b/bn/baseline/data_15000_1000/checkpoint-400/adapter_model/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..5444ff418752eb22d4d0b56630b185f30328a13c
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/adapter_model/adapter_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eaae1e886a57f5c5700fa5cc45e2d7e6aa73ffe52e8820fadf55c5c5cce179ec
+size 863
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/adapter_model/adapter_model.safetensors b/bn/baseline/data_15000_1000/checkpoint-400/adapter_model/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..14aced81d1a0ba6975695076582ea155ae3d4f55
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/adapter_model/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5b4a528a50374541cfd7eda52cca8c5f350f4ca60788c7468e88e6fbcb7769e
+size 42002584
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt b/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..19460130c0203dd6b3de9f234739a6fe5497d370
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d3acfea4f14aae6cb6e5a6f4424de2c4afa02d9134501a56cd22e011e554be3
+size 62930469
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt b/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..0ba3bfb44a99e1f4ea40ac76e287946f2186f63e
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8499a2a899e5823d242de0af3e53d746f654fe15813f247226248dc9c4281ee
+size 62930533
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt b/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..3115de019dc423056f5de30b1e17310b2e9f0b63
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d37cd0c3015b3f38bafa8403a028509377c157a29e96a2e68aa19c988764f5d8
+size 62930597
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt b/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..384c07598abbb83388f27a4ff3ac809222b0c4f7
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05045a211e8a629de7e142410260dba634b5f0bc1346fea781607d4e6f8a94db
+size 62930597
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt b/bn/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..9150977dc23d9f8c0b014e4a90c537fabb2bae47
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb2988ebca9f25bc8619d301dd96b5c32a5ab1ab6cb58b3aadeab99c050de0c6
+size 42291289
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/latest b/bn/baseline/data_15000_1000/checkpoint-400/latest
new file mode 100644
index 0000000000000000000000000000000000000000..e5bdf58d4f29d34e909da25905fad376f73e7c29
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/latest
@@ -0,0 +1 @@
+global_step400
\ No newline at end of file
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/rng_state_0.pth b/bn/baseline/data_15000_1000/checkpoint-400/rng_state_0.pth
new file mode 100644
index 0000000000000000000000000000000000000000..903f38e0f682e2aef736e3ea9a8a4ea7fbe3b261
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/rng_state_0.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9b014a9595b01563fb0e289117082dca7d7dd532a04bff217f706beec9eb6be
+size 15429
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/rng_state_1.pth b/bn/baseline/data_15000_1000/checkpoint-400/rng_state_1.pth
new file mode 100644
index 0000000000000000000000000000000000000000..eea49d9f12ac52efafefc051b0d5737f85e99ecf
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/rng_state_1.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:278c800e3b7bc583cb5f1453e1e796c0434617f55407c00bc43f50a3e35dc6da
+size 15429
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/rng_state_2.pth b/bn/baseline/data_15000_1000/checkpoint-400/rng_state_2.pth
new file mode 100644
index 0000000000000000000000000000000000000000..676160a80c06389377d8eaa0dcef0c8c46b6204c
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/rng_state_2.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0a62bd879d3de108c1560e1a8249dd98c0ef66ebcabdb8db14fec0e71fd173f
+size 15429
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/rng_state_3.pth b/bn/baseline/data_15000_1000/checkpoint-400/rng_state_3.pth
new file mode 100644
index 0000000000000000000000000000000000000000..b5b8bbb7a7da5a1963ea58f3fbf981df43e6e94f
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/rng_state_3.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98254fd6b5306b24a9b5f382c54b266292b9f780cd0e76743ed28395b2065943
+size 15429
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/special_tokens_map.json b/bn/baseline/data_15000_1000/checkpoint-400/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..4ed2bd8240878a7a0d4fd2c60cdc89f6d7a5f1e1
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/special_tokens_map.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83da1082df286d75a2984dc06ec439f4febc3d862ac55d199402e5d345f5773a
+size 372
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/tokenizer.json b/bn/baseline/data_15000_1000/checkpoint-400/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..66cd9d7e0daec95eb10d16a63c615637dbbb7304
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79e3e522635f3171300913bb421464a87de6222182a0570b9b2ccba2a964b2b4
+size 9085657
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/tokenizer_config.json b/bn/baseline/data_15000_1000/checkpoint-400/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..7144ada11807e90b92529f17434f8d01915c3dff
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/tokenizer_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d967a51bb800d3e471ea23dd3f7b054b136600238bbbb40612b8b96b0370746e
+size 55427
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/trainer_state.json b/bn/baseline/data_15000_1000/checkpoint-400/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..de72d490aeba67030da7037cf8a095271190df60
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/trainer_state.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35d4f2b1d905604f1e35e5c37779f058d62926b9188178d0024984e8c7833e5e
+size 69712
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/training_args.bin b/bn/baseline/data_15000_1000/checkpoint-400/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..75e2af92638c9ebdd3a0bab1aeafcb8d42d32cdb
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3acc7c6d4109909cd8d0a6012eabedc6936794f718ccb8e9dda270d768c9160
+size 7761
diff --git a/bn/baseline/data_15000_1000/checkpoint-400/zero_to_fp32.py b/bn/baseline/data_15000_1000/checkpoint-400/zero_to_fp32.py
new file mode 100755
index 0000000000000000000000000000000000000000..0e759146cadd92ddfefab3680146c2bd6a2b5c04
--- /dev/null
+++ b/bn/baseline/data_15000_1000/checkpoint-400/zero_to_fp32.py
@@ -0,0 +1,760 @@
+#!/usr/bin/env python
+
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: Apache-2.0
+
+# DeepSpeed Team
+
+# This script extracts fp32 consolidated weights from a zero 1, 2 and 3 DeepSpeed checkpoints. It gets
+# copied into the top level checkpoint dir, so the user can easily do the conversion at any point in
+# the future. Once extracted, the weights don't require DeepSpeed and can be used in any
+# application.
+#
+# example:
+#   python zero_to_fp32.py . output_dir/
+#   or
+#   python zero_to_fp32.py . output_dir/ --safe_serialization
+
+import argparse
+import torch
+import glob
+import math
+import os
+import re
+import gc
+import json
+import numpy as np
+from tqdm import tqdm
+from collections import OrderedDict
+from dataclasses import dataclass
+
+# while this script doesn't use deepspeed to recover data, since the checkpoints are pickled with
+# DeepSpeed data structures it has to be available in the current python environment.
+from deepspeed.utils import logger
+from deepspeed.checkpoint.constants import (DS_VERSION, OPTIMIZER_STATE_DICT, SINGLE_PARTITION_OF_FP32_GROUPS,
+                                            FP32_FLAT_GROUPS, ZERO_STAGE, PARTITION_COUNT, PARAM_SHAPES, BUFFER_NAMES,
+                                            FROZEN_PARAM_SHAPES, FROZEN_PARAM_FRAGMENTS)
+
+
+@dataclass
+class zero_model_state:
+    buffers: dict()
+    param_shapes: dict()
+    shared_params: list
+    ds_version: int
+    frozen_param_shapes: dict()
+    frozen_param_fragments: dict()
+
+
+debug = 0
+
+# load to cpu
+device = torch.device('cpu')
+
+
+def atoi(text):
+    return int(text) if text.isdigit() else text
+
+
+def natural_keys(text):
+    '''
+    alist.sort(key=natural_keys) sorts in human order
+    http://nedbatchelder.com/blog/200712/human_sorting.html
+    (See Toothy's implementation in the comments)
+    '''
+    return [atoi(c) for c in re.split(r'(\d+)', text)]
+
+
+def get_model_state_file(checkpoint_dir, zero_stage):
+    if not os.path.isdir(checkpoint_dir):
+        raise FileNotFoundError(f"Directory '{checkpoint_dir}' doesn't exist")
+
+    # there should be only one file
+    if zero_stage <= 2:
+        file = os.path.join(checkpoint_dir, "mp_rank_00_model_states.pt")
+    elif zero_stage == 3:
+        file = os.path.join(checkpoint_dir, "zero_pp_rank_0_mp_rank_00_model_states.pt")
+
+    if not os.path.exists(file):
+        raise FileNotFoundError(f"can't find model states file at '{file}'")
+
+    return file
+
+
+def get_checkpoint_files(checkpoint_dir, glob_pattern):
+    # XXX: need to test that this simple glob rule works for multi-node setup too
+    ckpt_files = sorted(glob.glob(os.path.join(checkpoint_dir, glob_pattern)), key=natural_keys)
+
+    if len(ckpt_files) == 0:
+        raise FileNotFoundError(f"can't find {glob_pattern} files in directory '{checkpoint_dir}'")
+
+    return ckpt_files
+
+
+def get_optim_files(checkpoint_dir):
+    return get_checkpoint_files(checkpoint_dir, "*_optim_states.pt")
+
+
+def get_model_state_files(checkpoint_dir):
+    return get_checkpoint_files(checkpoint_dir, "*_model_states.pt")
+
+
+def parse_model_states(files):
+    zero_model_states = []
+    for file in files:
+        state_dict = torch.load(file, map_location=device, weights_only=False)
+
+        if BUFFER_NAMES not in state_dict:
+            raise ValueError(f"{file} is not a model state checkpoint")
+        buffer_names = state_dict[BUFFER_NAMES]
+        if debug:
+            print("Found buffers:", buffer_names)
+
+        # recover just the buffers while restoring them to fp32 if they were saved in fp16
+        buffers = {k: v.float() for k, v in state_dict["module"].items() if k in buffer_names}
+        param_shapes = state_dict[PARAM_SHAPES]
+
+        # collect parameters that are included in param_shapes
+        param_names = []
+        for s in param_shapes:
+            for name in s.keys():
+                param_names.append(name)
+
+        # update with frozen parameters
+        frozen_param_shapes = state_dict.get(FROZEN_PARAM_SHAPES, None)
+        if frozen_param_shapes is not None:
+            if debug:
+                print(f"Found frozen_param_shapes: {frozen_param_shapes}")
+            param_names += list(frozen_param_shapes.keys())
+
+        # handle shared params
+        shared_params = [[k, v] for k, v in state_dict["shared_params"].items()]
+
+        ds_version = state_dict.get(DS_VERSION, None)
+
+        frozen_param_fragments = state_dict.get(FROZEN_PARAM_FRAGMENTS, None)
+
+        z_model_state = zero_model_state(buffers=buffers,
+                                         param_shapes=param_shapes,
+                                         shared_params=shared_params,
+                                         ds_version=ds_version,
+                                         frozen_param_shapes=frozen_param_shapes,
+                                         frozen_param_fragments=frozen_param_fragments)
+        zero_model_states.append(z_model_state)
+
+    return zero_model_states
+
+
+def parse_optim_states(files, ds_checkpoint_dir):
+    total_files = len(files)
+    state_dicts = []
+    for f in tqdm(files, desc='Loading checkpoint shards'):
+        state_dict = torch.load(f, map_location=device, mmap=True, weights_only=False)
+        # immediately discard the potentially huge 2 optimizer states as we only care for fp32 master weights
+        # and also handle the case where it was already removed by another helper script
+        state_dict["optimizer_state_dict"].pop("optimizer_state_dict", None)
+        state_dicts.append(state_dict)
+
+    if not ZERO_STAGE in state_dicts[0][OPTIMIZER_STATE_DICT]:
+        raise ValueError(f"{files[0]} is not a zero checkpoint")
+    zero_stage = state_dicts[0][OPTIMIZER_STATE_DICT][ZERO_STAGE]
+    world_size = state_dicts[0][OPTIMIZER_STATE_DICT][PARTITION_COUNT]
+
+    # For ZeRO-2 each param group can have different partition_count as data parallelism for expert
+    # parameters can be different from data parallelism for non-expert parameters. So we can just
+    # use the max of the partition_count to get the dp world_size.
+
+    if type(world_size) is list:
+        world_size = max(world_size)
+
+    if world_size != total_files:
+        raise ValueError(
+            f"Expected {world_size} of '*_optim_states.pt' under '{ds_checkpoint_dir}' but found {total_files} files. "
+            "Possibly due to an overwrite of an old checkpoint, or a checkpoint didn't get saved by one or more processes."
+        )
+
+    # the groups are named differently in each stage
+    if zero_stage <= 2:
+        fp32_groups_key = SINGLE_PARTITION_OF_FP32_GROUPS
+    elif zero_stage == 3:
+        fp32_groups_key = FP32_FLAT_GROUPS
+    else:
+        raise ValueError(f"unknown zero stage {zero_stage}")
+
+    fp32_flat_groups = [state_dicts[i][OPTIMIZER_STATE_DICT][fp32_groups_key] for i in range(len(state_dicts))]
+    return zero_stage, world_size, fp32_flat_groups
+
+
+def _get_fp32_state_dict_from_zero_checkpoint(ds_checkpoint_dir, exclude_frozen_parameters):
+    """
+    Returns fp32 state_dict reconstructed from ds checkpoint
+
+    Args:
+        - ``ds_checkpoint_dir``: path to the deepspeed checkpoint folder (where the optimizer files are)
+
+    """
+    print(f"Processing zero checkpoint '{ds_checkpoint_dir}'")
+
+    optim_files = get_optim_files(ds_checkpoint_dir)
+    zero_stage, world_size, fp32_flat_groups = parse_optim_states(optim_files, ds_checkpoint_dir)
+    print(f"Detected checkpoint of type zero stage {zero_stage}, world_size: {world_size}")
+
+    model_files = get_model_state_files(ds_checkpoint_dir)
+
+    zero_model_states = parse_model_states(model_files)
+    print(f'Parsing checkpoint created by deepspeed=={zero_model_states[0].ds_version}')
+
+    if zero_stage <= 2:
+        return _get_fp32_state_dict_from_zero2_checkpoint(world_size, fp32_flat_groups, zero_model_states,
+                                                          exclude_frozen_parameters)
+    elif zero_stage == 3:
+        return _get_fp32_state_dict_from_zero3_checkpoint(world_size, fp32_flat_groups, zero_model_states,
+                                                          exclude_frozen_parameters)
+
+
+def _zero2_merge_frozen_params(state_dict, zero_model_states):
+    if zero_model_states[0].frozen_param_shapes is None or len(zero_model_states[0].frozen_param_shapes) == 0:
+        return
+
+    frozen_param_shapes = zero_model_states[0].frozen_param_shapes
+    frozen_param_fragments = zero_model_states[0].frozen_param_fragments
+
+    if debug:
+        num_elem = sum(s.numel() for s in frozen_param_shapes.values())
+        print(f'rank 0: {FROZEN_PARAM_SHAPES}.numel = {num_elem}')
+
+        wanted_params = len(frozen_param_shapes)
+        wanted_numel = sum(s.numel() for s in frozen_param_shapes.values())
+        avail_numel = sum([p.numel() for p in frozen_param_fragments.values()])
+        print(f'Frozen params: Have {avail_numel} numels to process.')
+        print(f'Frozen params: Need {wanted_numel} numels in {wanted_params} params')
+
+    total_params = 0
+    total_numel = 0
+    for name, shape in frozen_param_shapes.items():
+        total_params += 1
+        unpartitioned_numel = shape.numel()
+        total_numel += unpartitioned_numel
+
+        state_dict[name] = frozen_param_fragments[name]
+
+        if debug:
+            print(f"{name} full shape: {shape} unpartitioned numel {unpartitioned_numel} ")
+
+    print(f"Reconstructed Frozen fp32 state dict with {total_params} params {total_numel} elements")
+
+
+def _has_callable(obj, fn):
+    attr = getattr(obj, fn, None)
+    return callable(attr)
+
+
+def _zero2_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states):
+    param_shapes = zero_model_states[0].param_shapes
+
+    # Reconstruction protocol:
+    #
+    # XXX: document this
+
+    if debug:
+        for i in range(world_size):
+            for j in range(len(fp32_flat_groups[0])):
+                print(f"{FP32_FLAT_GROUPS}[{i}][{j}].shape={fp32_flat_groups[i][j].shape}")
+
+    # XXX: memory usage doubles here (zero2)
+    num_param_groups = len(fp32_flat_groups[0])
+    merged_single_partition_of_fp32_groups = []
+    for i in range(num_param_groups):
+        merged_partitions = [sd[i] for sd in fp32_flat_groups]
+        full_single_fp32_vector = torch.cat(merged_partitions, 0)
+        merged_single_partition_of_fp32_groups.append(full_single_fp32_vector)
+    avail_numel = sum(
+        [full_single_fp32_vector.numel() for full_single_fp32_vector in merged_single_partition_of_fp32_groups])
+
+    if debug:
+        wanted_params = sum([len(shapes) for shapes in param_shapes])
+        wanted_numel = sum([sum(shape.numel() for shape in shapes.values()) for shapes in param_shapes])
+        # not asserting if there is a mismatch due to possible padding
+        print(f"Have {avail_numel} numels to process.")
+        print(f"Need {wanted_numel} numels in {wanted_params} params.")
+
+    # params
+    # XXX: for huge models that can't fit into the host's RAM we will have to recode this to support
+    # out-of-core computing solution
+    total_numel = 0
+    total_params = 0
+    for shapes, full_single_fp32_vector in zip(param_shapes, merged_single_partition_of_fp32_groups):
+        offset = 0
+        avail_numel = full_single_fp32_vector.numel()
+        for name, shape in shapes.items():
+
+            unpartitioned_numel = shape.numel() if _has_callable(shape, 'numel') else math.prod(shape)
+            total_numel += unpartitioned_numel
+            total_params += 1
+
+            if debug:
+                print(f"{name} full shape: {shape} unpartitioned numel {unpartitioned_numel} ")
+            state_dict[name] = full_single_fp32_vector.narrow(0, offset, unpartitioned_numel).view(shape)
+            offset += unpartitioned_numel
+
+        # Z2 started to align to 2*world_size to improve nccl performance. Therefore both offset and
+        # avail_numel can differ by anywhere between 0..2*world_size. Due to two unrelated complex
+        # paddings performed in the code it's almost impossible to predict the exact numbers w/o the
+        # live optimizer object, so we are checking that the numbers are within the right range
+        align_to = 2 * world_size
+
+        def zero2_align(x):
+            return align_to * math.ceil(x / align_to)
+
+        if debug:
+            print(f"original offset={offset}, avail_numel={avail_numel}")
+
+        offset = zero2_align(offset)
+        avail_numel = zero2_align(avail_numel)
+
+        if debug:
+            print(f"aligned  offset={offset}, avail_numel={avail_numel}")
+
+        # Sanity check
+        if offset != avail_numel:
+            raise ValueError(f"consumed {offset} numels out of {avail_numel} - something is wrong")
+
+    print(f"Reconstructed fp32 state dict with {total_params} params {total_numel} elements")
+
+
+def _get_fp32_state_dict_from_zero2_checkpoint(world_size, fp32_flat_groups, zero_model_states,
+                                               exclude_frozen_parameters):
+    state_dict = OrderedDict()
+
+    # buffers
+    buffers = zero_model_states[0].buffers
+    state_dict.update(buffers)
+    if debug:
+        print(f"added {len(buffers)} buffers")
+
+    if not exclude_frozen_parameters:
+        _zero2_merge_frozen_params(state_dict, zero_model_states)
+
+    _zero2_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states)
+
+    # recover shared parameters
+    for pair in zero_model_states[0].shared_params:
+        if pair[1] in state_dict:
+            state_dict[pair[0]] = state_dict[pair[1]]
+
+    return state_dict
+
+
+def zero3_partitioned_param_info(unpartitioned_numel, world_size):
+    remainder = unpartitioned_numel % world_size
+    padding_numel = (world_size - remainder) if remainder else 0
+    partitioned_numel = math.ceil(unpartitioned_numel / world_size)
+    return partitioned_numel, padding_numel
+
+
+def _zero3_merge_frozen_params(state_dict, world_size, zero_model_states):
+    if zero_model_states[0].frozen_param_shapes is None or len(zero_model_states[0].frozen_param_shapes) == 0:
+        return
+
+    if debug:
+        for i in range(world_size):
+            num_elem = sum(s.numel() for s in zero_model_states[i].frozen_param_fragments.values())
+            print(f'rank {i}: {FROZEN_PARAM_SHAPES}.numel = {num_elem}')
+
+        frozen_param_shapes = zero_model_states[0].frozen_param_shapes
+        wanted_params = len(frozen_param_shapes)
+        wanted_numel = sum(s.numel() for s in frozen_param_shapes.values())
+        avail_numel = sum([p.numel() for p in zero_model_states[0].frozen_param_fragments.values()]) * world_size
+        print(f'Frozen params: Have {avail_numel} numels to process.')
+        print(f'Frozen params: Need {wanted_numel} numels in {wanted_params} params')
+
+    total_params = 0
+    total_numel = 0
+    for name, shape in zero_model_states[0].frozen_param_shapes.items():
+        total_params += 1
+        unpartitioned_numel = shape.numel()
+        total_numel += unpartitioned_numel
+
+        param_frags = tuple(model_state.frozen_param_fragments[name] for model_state in zero_model_states)
+        state_dict[name] = torch.cat(param_frags, 0).narrow(0, 0, unpartitioned_numel).view(shape)
+
+        partitioned_numel, partitioned_padding_numel = zero3_partitioned_param_info(unpartitioned_numel, world_size)
+
+        if debug:
+            print(
+                f"Frozen params: {total_params} {name} full shape: {shape} partition0 numel={partitioned_numel} partitioned_padding_numel={partitioned_padding_numel}"
+            )
+
+    print(f"Reconstructed Frozen fp32 state dict with {total_params} params {total_numel} elements")
+
+
+class GatheredTensor:
+    """
+    A pseudo tensor that collects partitioned weights.
+    It is more memory efficient when there are multiple groups.
+    """
+
+    def __init__(self, flat_groups, flat_groups_offset, offset, partitioned_numel, shape):
+        self.flat_groups = flat_groups
+        self.flat_groups_offset = flat_groups_offset
+        self.offset = offset
+        self.partitioned_numel = partitioned_numel
+        self.shape = shape
+        self.dtype = self.flat_groups[0][0].dtype
+
+    def contiguous(self):
+        """
+        Merge partitioned weights from flat_groups into a single tensor.
+        """
+        end_idx = self.offset + self.partitioned_numel
+        world_size = len(self.flat_groups)
+        pad_flat_param_chunks = []
+
+        for rank_i in range(world_size):
+            # for each rank, we need to collect weights from related group/groups
+            flat_groups_at_rank_i = self.flat_groups[rank_i]
+            start_group_id = None
+            end_group_id = None
+            for group_id in range(len(self.flat_groups_offset)):
+                if self.flat_groups_offset[group_id] <= self.offset < self.flat_groups_offset[group_id + 1]:
+                    start_group_id = group_id
+                if self.flat_groups_offset[group_id] < end_idx <= self.flat_groups_offset[group_id + 1]:
+                    end_group_id = group_id
+                    break
+            # collect weights from related group/groups
+            for group_id in range(start_group_id, end_group_id + 1):
+                flat_tensor = flat_groups_at_rank_i[group_id]
+                start_offset = self.offset - self.flat_groups_offset[group_id]
+                end_offset = min(end_idx, self.flat_groups_offset[group_id + 1]) - self.flat_groups_offset[group_id]
+                pad_flat_param_chunks.append(flat_tensor[start_offset:end_offset])
+
+        # collect weights from all ranks
+        pad_flat_param = torch.cat(pad_flat_param_chunks, dim=0)
+        param = pad_flat_param[:self.shape.numel()].view(self.shape).contiguous()
+        return param
+
+
+def _zero3_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states):
+    param_shapes = zero_model_states[0].param_shapes
+    avail_numel = sum([flat_group.numel() for flat_group in fp32_flat_groups[0]]) * world_size
+
+    # Reconstruction protocol: For zero3 we need to zip the partitions together at boundary of each
+    # param, re-consolidating each param, while dealing with padding if any
+
+    # merge list of dicts, preserving order
+    param_shapes = {k: v for d in param_shapes for k, v in d.items()}
+
+    if debug:
+        for i in range(world_size):
+            print(f"{FP32_FLAT_GROUPS}[{i}].shape={fp32_flat_groups[i].shape}")
+
+        wanted_params = len(param_shapes)
+        wanted_numel = sum(shape.numel() for shape in param_shapes.values())
+        # not asserting if there is a mismatch due to possible padding
+        avail_numel = fp32_flat_groups[0].numel() * world_size
+        print(f"Trainable params: Have {avail_numel} numels to process.")
+        print(f"Trainable params: Need {wanted_numel} numels in {wanted_params} params.")
+
+    # params
+    # XXX: for huge models that can't fit into the host's RAM we will have to recode this to support
+    # out-of-core computing solution
+    offset = 0
+    total_numel = 0
+    total_params = 0
+    flat_groups_offset = [0] + list(np.cumsum([flat_tensor.numel() for flat_tensor in fp32_flat_groups[0]]))
+    for name, shape in tqdm(param_shapes.items(), desc='Gathering sharded weights'):
+        unpartitioned_numel = shape.numel()
+        total_numel += unpartitioned_numel
+        total_params += 1
+        partitioned_numel, partitioned_padding_numel = zero3_partitioned_param_info(unpartitioned_numel, world_size)
+
+        if debug:
+            print(
+                f"Trainable params: {total_params} {name} full shape: {shape} partition0 numel={partitioned_numel} partitioned_padding_numel={partitioned_padding_numel}"
+            )
+
+        # memory efficient tensor
+        tensor = GatheredTensor(fp32_flat_groups, flat_groups_offset, offset, partitioned_numel, shape)
+        state_dict[name] = tensor
+        offset += partitioned_numel
+
+    offset *= world_size
+
+    # Sanity check
+    if offset != avail_numel:
+        raise ValueError(f"consumed {offset} numels out of {avail_numel} - something is wrong")
+
+    print(f"Reconstructed Trainable fp32 state dict with {total_params} params {total_numel} elements")
+
+
+def _get_fp32_state_dict_from_zero3_checkpoint(world_size, fp32_flat_groups, zero_model_states,
+                                               exclude_frozen_parameters):
+    state_dict = OrderedDict()
+
+    # buffers
+    buffers = zero_model_states[0].buffers
+    state_dict.update(buffers)
+    if debug:
+        print(f"added {len(buffers)} buffers")
+
+    if not exclude_frozen_parameters:
+        _zero3_merge_frozen_params(state_dict, world_size, zero_model_states)
+
+    _zero3_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states)
+
+    # recover shared parameters
+    for pair in zero_model_states[0].shared_params:
+        if pair[1] in state_dict:
+            state_dict[pair[0]] = state_dict[pair[1]]
+
+    return state_dict
+
+
+def to_torch_tensor(state_dict, return_empty_tensor=False):
+    """
+    Convert state_dict of GatheredTensor to torch tensor
+    """
+    torch_state_dict = {}
+    converted_tensors = {}
+    for name, tensor in state_dict.items():
+        tensor_id = id(tensor)
+        if tensor_id in converted_tensors:  # shared tensors
+            shared_tensor = torch_state_dict[converted_tensors[tensor_id]]
+            torch_state_dict[name] = shared_tensor
+        else:
+            converted_tensors[tensor_id] = name
+            if return_empty_tensor:
+                torch_state_dict[name] = torch.empty(tensor.shape, dtype=tensor.dtype)
+            else:
+                torch_state_dict[name] = tensor.contiguous()
+    return torch_state_dict
+
+
+def get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir,
+                                             tag=None,
+                                             exclude_frozen_parameters=False,
+                                             lazy_mode=False):
+    """
+    Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated state_dict that can be loaded with
+    ``load_state_dict()`` and used for training without DeepSpeed or shared with others, for example
+    via a model hub.
+
+    Args:
+        - ``checkpoint_dir``: path to the desired checkpoint folder
+        - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in 'latest' file. e.g., ``global_step14``
+        - ``exclude_frozen_parameters``: exclude frozen parameters
+        - ``lazy_mode``: get state_dict in lazy mode. It returns a dict of pesduo tensor instead of torch tensor, which is more memory efficient.
+          Convert the pesduo tensor to torch tensor by ``.contiguous()``
+
+    Returns:
+        - pytorch ``state_dict``
+
+    A typical usage might be ::
+
+        from deepspeed.utils.zero_to_fp32 import get_fp32_state_dict_from_zero_checkpoint
+        # do the training and checkpoint saving
+        state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir) # already on cpu
+        model = model.cpu() # move to cpu
+        model.load_state_dict(state_dict)
+        # submit to model hub or save the model to share with others
+
+    In this example the ``model`` will no longer be usable in the deepspeed context of the same
+    application. i.e. you will need to re-initialize the deepspeed engine, since
+    ``model.load_state_dict(state_dict)`` will remove all the deepspeed magic from it.
+
+    If you want it all done for you, use ``load_state_dict_from_zero_checkpoint`` instead.
+
+    Note: the above usage may not work if your application doesn't have sufficient free CPU memory.
+    You may need to use the offline approach using the ``zero_to_fp32.py`` script that is saved with
+    the checkpoint. Or you can load state_dict in lazy mode ::
+
+        from deepspeed.utils.zero_to_fp32 import get_fp32_state_dict_from_zero_checkpoint
+        state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir, lazy_mode=True) # not on cpu
+        for name, lazy_tensor in state_dict.item():
+            tensor = lazy_tensor.contiguous()  # to cpu
+            print(name, tensor)
+            # del tensor to release memory if it no longer in use
+    """
+    if tag is None:
+        latest_path = os.path.join(checkpoint_dir, 'latest')
+        if os.path.isfile(latest_path):
+            with open(latest_path, 'r') as fd:
+                tag = fd.read().strip()
+        else:
+            raise ValueError(f"Unable to find 'latest' file at {latest_path}")
+
+    ds_checkpoint_dir = os.path.join(checkpoint_dir, tag)
+
+    if not os.path.isdir(ds_checkpoint_dir):
+        raise FileNotFoundError(f"Directory '{ds_checkpoint_dir}' doesn't exist")
+
+    state_dict = _get_fp32_state_dict_from_zero_checkpoint(ds_checkpoint_dir, exclude_frozen_parameters)
+    if lazy_mode:
+        return state_dict
+    else:
+        return to_torch_tensor(state_dict)
+
+
+def convert_zero_checkpoint_to_fp32_state_dict(checkpoint_dir,
+                                               output_dir,
+                                               max_shard_size="5GB",
+                                               safe_serialization=False,
+                                               tag=None,
+                                               exclude_frozen_parameters=False):
+    """
+    Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated ``state_dict`` file that can be
+    loaded with ``torch.load(file)`` + ``load_state_dict()`` and used for training without DeepSpeed.
+
+    Args:
+        - ``checkpoint_dir``: path to the desired checkpoint folder. (one that contains the tag-folder, like ``global_step14``)
+        - ``output_dir``: directory to the pytorch fp32 state_dict output files
+        - ``max_shard_size``: the maximum size for a checkpoint before being sharded, default value is 5GB
+        - ``safe_serialization``:  whether to save the model using `safetensors` or the traditional PyTorch way (that uses `pickle`).
+        - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in the file named ``latest`` in the checkpoint folder, e.g., ``global_step14``
+        - ``exclude_frozen_parameters``: exclude frozen parameters
+    """
+
+    # Dependency pre-check
+    if safe_serialization:
+        try:
+            from safetensors.torch import save_file
+        except ImportError:
+            print('If you want to use `safe_serialization`, please `pip install safetensors`')
+            raise
+    if max_shard_size is not None:
+        try:
+            from huggingface_hub import split_torch_state_dict_into_shards
+        except ImportError:
+            print('If you want to use `max_shard_size`, please `pip install huggingface_hub`')
+            raise
+
+    # Convert zero checkpoint to state_dict
+    state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir,
+                                                          tag,
+                                                          exclude_frozen_parameters,
+                                                          lazy_mode=True)
+
+    # Shard the model if it is too big.
+    weights_name = "model.safetensors" if safe_serialization else "pytorch_model.bin"
+    if max_shard_size is not None:
+        filename_pattern = weights_name.replace(".bin", "{suffix}.bin").replace(".safetensors", "{suffix}.safetensors")
+        # an memory-efficient approach for sharding
+        empty_state_dict = to_torch_tensor(state_dict, return_empty_tensor=True)
+        state_dict_split = split_torch_state_dict_into_shards(empty_state_dict,
+                                                              filename_pattern=filename_pattern,
+                                                              max_shard_size=max_shard_size)
+    else:
+        from collections import namedtuple
+        StateDictSplit = namedtuple("StateDictSplit", ["is_sharded", "filename_to_tensors"])
+        state_dict_split = StateDictSplit(is_sharded=False,
+                                          filename_to_tensors={weights_name: list(state_dict.keys())})
+
+    # Save the model by shard
+    os.makedirs(output_dir, exist_ok=True)
+    filename_to_tensors = state_dict_split.filename_to_tensors.items()
+    for shard_file, tensors in tqdm(filename_to_tensors, desc="Saving checkpoint shards"):
+        shard_state_dict = {tensor_name: state_dict[tensor_name] for tensor_name in tensors}
+        shard_state_dict = to_torch_tensor(shard_state_dict)
+        output_path = os.path.join(output_dir, shard_file)
+        if safe_serialization:
+            save_file(shard_state_dict, output_path, metadata={"format": "pt"})
+        else:
+            torch.save(shard_state_dict, output_path)
+        # release the memory of current shard
+        for tensor_name in list(shard_state_dict.keys()):
+            del state_dict[tensor_name]
+            del shard_state_dict[tensor_name]
+        del shard_state_dict
+        gc.collect()
+
+    # Save index if sharded
+    if state_dict_split.is_sharded:
+        index = {
+            "metadata": state_dict_split.metadata,
+            "weight_map": state_dict_split.tensor_to_filename,
+        }
+        save_index_file = "model.safetensors.index.json" if safe_serialization else "pytorch_model.bin.index.json"
+        save_index_file = os.path.join(output_dir, save_index_file)
+        with open(save_index_file, "w", encoding="utf-8") as f:
+            content = json.dumps(index, indent=2, sort_keys=True) + "\n"
+            f.write(content)
+
+
+def load_state_dict_from_zero_checkpoint(model, checkpoint_dir, tag=None):
+    """
+    1. Put the provided model to cpu
+    2. Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated ``state_dict``
+    3. Load it into the provided model
+
+    Args:
+        - ``model``: the model object to update
+        - ``checkpoint_dir``: path to the desired checkpoint folder. (one that contains the tag-folder, like ``global_step14``)
+        - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in the file named ``latest`` in the checkpoint folder, e.g., ``global_step14``
+
+    Returns:
+        - ``model`: modified model
+
+    Make sure you have plenty of CPU memory available before you call this function. If you don't
+    have enough use the ``zero_to_fp32.py`` utility to do the conversion. You will find it
+    conveniently placed for you in the checkpoint folder.
+
+    A typical usage might be ::
+
+        from deepspeed.utils.zero_to_fp32 import load_state_dict_from_zero_checkpoint
+        model = load_state_dict_from_zero_checkpoint(trainer.model, checkpoint_dir)
+        # submit to model hub or save the model to share with others
+
+    Note, that once this was run, the ``model`` will no longer be usable in the deepspeed context
+    of the same application. i.e. you will need to re-initialize the deepspeed engine, since
+    ``model.load_state_dict(state_dict)`` will remove all the deepspeed magic from it.
+
+    """
+    logger.info(f"Extracting fp32 weights")
+    state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir, tag)
+
+    logger.info(f"Overwriting model with fp32 weights")
+    model = model.cpu()
+    model.load_state_dict(state_dict, strict=False)
+
+    return model
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("checkpoint_dir",
+                        type=str,
+                        help="path to the desired checkpoint folder, e.g., path/checkpoint-12")
+    parser.add_argument("output_dir",
+                        type=str,
+                        help="directory to the pytorch fp32 state_dict output files"
+                        "(e.g. path/checkpoint-12-output/)")
+    parser.add_argument(
+        "--max_shard_size",
+        type=str,
+        default="5GB",
+        help="The maximum size for a checkpoint before being sharded. Checkpoints shard will then be each of size"
+        "lower than this size. If expressed as a string, needs to be digits followed by a unit (like `5MB`"
+        "We default it to 5GB in order for models to be able to run easily on free-tier google colab instances"
+        "without CPU OOM issues.")
+    parser.add_argument(
+        "--safe_serialization",
+        default=False,
+        action='store_true',
+        help="Whether to save the model using `safetensors` or the traditional PyTorch way (that uses `pickle`).")
+    parser.add_argument("-t",
+                        "--tag",
+                        type=str,
+                        default=None,
+                        help="checkpoint tag used as a unique identifier for checkpoint. e.g., global_step1")
+    parser.add_argument("--exclude_frozen_parameters", action='store_true', help="exclude frozen parameters")
+    parser.add_argument("-d", "--debug", action='store_true', help="enable debug")
+    args = parser.parse_args()
+
+    debug = args.debug
+
+    convert_zero_checkpoint_to_fp32_state_dict(args.checkpoint_dir,
+                                               args.output_dir,
+                                               max_shard_size=args.max_shard_size,
+                                               safe_serialization=args.safe_serialization,
+                                               tag=args.tag,
+                                               exclude_frozen_parameters=args.exclude_frozen_parameters)
diff --git a/bn/baseline/data_15000_1000/eval_results.json b/bn/baseline/data_15000_1000/eval_results.json
new file mode 100644
index 0000000000000000000000000000000000000000..5767372dddfd25f9963e31f3d8a280801119615c
--- /dev/null
+++ b/bn/baseline/data_15000_1000/eval_results.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:456c8bc35265ada9ecc44cfe4234bf1522e03db7fdb8a6fdd2f8c9f2bd619f6a
+size 268
diff --git a/bn/baseline/data_15000_1000/special_tokens_map.json b/bn/baseline/data_15000_1000/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..4ed2bd8240878a7a0d4fd2c60cdc89f6d7a5f1e1
--- /dev/null
+++ b/bn/baseline/data_15000_1000/special_tokens_map.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83da1082df286d75a2984dc06ec439f4febc3d862ac55d199402e5d345f5773a
+size 372
diff --git a/bn/baseline/data_15000_1000/tokenizer.json b/bn/baseline/data_15000_1000/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..66cd9d7e0daec95eb10d16a63c615637dbbb7304
--- /dev/null
+++ b/bn/baseline/data_15000_1000/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79e3e522635f3171300913bb421464a87de6222182a0570b9b2ccba2a964b2b4
+size 9085657
diff --git a/bn/baseline/data_15000_1000/tokenizer_config.json b/bn/baseline/data_15000_1000/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..7144ada11807e90b92529f17434f8d01915c3dff
--- /dev/null
+++ b/bn/baseline/data_15000_1000/tokenizer_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d967a51bb800d3e471ea23dd3f7b054b136600238bbbb40612b8b96b0370746e
+size 55427
diff --git a/bn/baseline/data_15000_1000/train.log b/bn/baseline/data_15000_1000/train.log
new file mode 100644
index 0000000000000000000000000000000000000000..4dcd1b0a9235f4e3d8ec4676ce5d377b8cf469b6
--- /dev/null
+++ b/bn/baseline/data_15000_1000/train.log
@@ -0,0 +1,2145 @@
+W0626 23:48:19.327176 1412956 site-packages/torch/distributed/run.py:766] 
+W0626 23:48:19.327176 1412956 site-packages/torch/distributed/run.py:766] *****************************************
+W0626 23:48:19.327176 1412956 site-packages/torch/distributed/run.py:766] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+W0626 23:48:19.327176 1412956 site-packages/torch/distributed/run.py:766] *****************************************
+[2025-06-26 23:48:25,050] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-26 23:48:25,125] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-26 23:48:25,167] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-26 23:48:25,168] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-26 23:48:26,605] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-26 23:48:26,715] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-26 23:48:26,733] [INFO] [comm.py:675:init_distributed] cdb=None
+[2025-06-26 23:48:26,733] [INFO] [comm.py:706:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
+[2025-06-26 23:48:26,748] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-26 23:48:26,778] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-26 23:48:26,847] [INFO] [comm.py:675:init_distributed] cdb=None
+[2025-06-26 23:48:26,880] [INFO] [comm.py:675:init_distributed] cdb=None
+[2025-06-26 23:48:26,915] [INFO] [comm.py:675:init_distributed] cdb=None
+06/26/2025 23:48:27 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1distributed training: True, 16-bits training: False
+06/26/2025 23:48:27 - INFO - __main__ - Training/evaluation parameters LoRATrainingArguments(
+_n_gpu=1,
+accelerator_config={'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None, 'use_configured_state': False},
+adafactor=False,
+adam_beta1=0.9,
+adam_beta2=0.999,
+adam_epsilon=1e-08,
+auto_find_batch_size=False,
+batch_eval_metrics=False,
+bf16=True,
+bf16_full_eval=True,
+data_seed=None,
+dataloader_drop_last=False,
+dataloader_num_workers=2,
+dataloader_persistent_workers=False,
+dataloader_pin_memory=True,
+dataloader_prefetch_factor=None,
+ddp_backend=None,
+ddp_broadcast_buffers=None,
+ddp_bucket_cap_mb=None,
+ddp_find_unused_parameters=None,
+ddp_timeout=3600,
+debug=[],
+deepspeed=./config/deepspeed_config.json,
+disable_tqdm=False,
+dispatch_batches=None,
+do_eval=True,
+do_predict=False,
+do_train=True,
+eval_accumulation_steps=None,
+eval_delay=0,
+eval_do_concat_batches=True,
+eval_on_start=True,
+eval_steps=200,
+eval_strategy=steps,
+eval_use_gather_object=False,
+evaluation_strategy=None,
+fp16=False,
+fp16_backend=auto,
+fp16_full_eval=False,
+fp16_opt_level=O1,
+fsdp=[],
+fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False},
+fsdp_min_num_params=0,
+fsdp_transformer_layer_cls_to_wrap=None,
+full_determinism=False,
+gradient_accumulation_steps=1,
+gradient_checkpointing=True,
+gradient_checkpointing_kwargs=None,
+greater_is_better=False,
+group_by_length=False,
+half_precision_backend=auto,
+hub_always_push=False,
+hub_model_id=None,
+hub_private_repo=False,
+hub_strategy=every_save,
+hub_token=<HUB_TOKEN>,
+ignore_data_skip=False,
+include_inputs_for_metrics=False,
+include_num_input_tokens_seen=False,
+include_tokens_per_second=False,
+jit_mode_eval=False,
+label_names=None,
+label_smoothing_factor=0.0,
+learning_rate=0.0005,
+length_column_name=length,
+load_best_model_at_end=True,
+load_lora_from=None,
+local_rank=0,
+log_level=passive,
+log_level_replica=warning,
+log_on_each_node=True,
+logging_dir=./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/runs/Jun26_23-48-24_innmi1srh2-p040,
+logging_first_step=False,
+logging_nan_inf_filter=True,
+logging_steps=1.0,
+logging_strategy=steps,
+lora_config=./config/lora_config.json,
+lr_scheduler_kwargs={},
+lr_scheduler_type=inverse_sqrt,
+max_grad_norm=1.0,
+max_steps=-1,
+metric_for_best_model=eval_loss,
+mp_parameters=,
+neftune_noise_alpha=None,
+no_cuda=False,
+num_train_epochs=5.0,
+optim=adamw_torch,
+optim_args=None,
+optim_target_modules=None,
+output_dir=./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/,
+overwrite_output_dir=True,
+past_index=-1,
+per_device_eval_batch_size=25,
+per_device_train_batch_size=25,
+prediction_loss_only=False,
+push_to_hub=False,
+push_to_hub_model_id=None,
+push_to_hub_organization=None,
+push_to_hub_token=<PUSH_TO_HUB_TOKEN>,
+ray_scope=last,
+remove_unused_columns=True,
+report_to=['wandb'],
+restore_callback_states_from_checkpoint=False,
+resume_from_checkpoint=None,
+run_name=./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/,
+save_on_each_node=False,
+save_only_model=False,
+save_safetensors=True,
+save_steps=200,
+save_strategy=steps,
+save_total_limit=1,
+seed=1,
+skip_memory_metrics=True,
+split_batches=None,
+tf32=None,
+torch_compile=False,
+torch_compile_backend=None,
+torch_compile_mode=None,
+torch_empty_cache_steps=None,
+torchdynamo=None,
+tpu_metrics_debug=False,
+tpu_num_cores=None,
+use_cpu=False,
+use_int8_training=False,
+use_ipex=False,
+use_legacy_prediction_loop=False,
+use_lora=True,
+use_mps_device=False,
+warmup_ratio=0.03,
+warmup_steps=0,
+weight_decay=0.0,
+)
+06/26/2025 23:48:27 - WARNING - __main__ - Process rank: 3, device: cuda:3, n_gpu: 1distributed training: True, 16-bits training: False
+06/26/2025 23:48:27 - WARNING - __main__ - Process rank: 2, device: cuda:2, n_gpu: 1distributed training: True, 16-bits training: False
+06/26/2025 23:48:27 - WARNING - __main__ - Process rank: 1, device: cuda:1, n_gpu: 1distributed training: True, 16-bits training: False
+Using custom data configuration default-10eaf7c5c1c6f11a
+06/26/2025 23:48:27 - INFO - datasets.builder - Using custom data configuration default-10eaf7c5c1c6f11a
+Loading Dataset Infos from /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/datasets/packaged_modules/json
+06/26/2025 23:48:27 - INFO - datasets.info - Loading Dataset Infos from /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/datasets/packaged_modules/json
+Overwrite dataset info from restored data version if exists.
+06/26/2025 23:48:27 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists.
+Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-10eaf7c5c1c6f11a/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+06/26/2025 23:48:27 - INFO - datasets.info - Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-10eaf7c5c1c6f11a/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+Found cached dataset json (/home/iitm_admin/.cache/huggingface/datasets/json/default-10eaf7c5c1c6f11a/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092)
+06/26/2025 23:48:27 - INFO - datasets.builder - Found cached dataset json (/home/iitm_admin/.cache/huggingface/datasets/json/default-10eaf7c5c1c6f11a/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092)
+Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-10eaf7c5c1c6f11a/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+06/26/2025 23:48:27 - INFO - datasets.info - Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-10eaf7c5c1c6f11a/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+[INFO|configuration_utils.py:733] 2025-06-26 23:48:28,434 >> loading configuration file config.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/config.json
+[INFO|configuration_utils.py:821] 2025-06-26 23:48:28,435 >> Model config LlamaConfig {
+  "_name_or_path": "meta-llama/Llama-3.1-8B-Instruct",
+  "additional_loss_layer": 16,
+  "alignment_matrices_path": null,
+  "apply_inverse": false,
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "contrastive_loss_temperature": 1.0,
+  "contrastive_loss_weight": 1.0,
+  "contrastive_pooling_type": "mean",
+  "distance_function": "cosine",
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "inject_Ws": false,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "only_train_contrastive": false,
+  "only_train_language_modeling": true,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.44.0.dev0",
+  "unidirectional_contrastive_loss": false,
+  "use_cache": true,
+  "vocab_size": 128256
+}
+
+[INFO|tokenization_utils_base.py:2269] 2025-06-26 23:48:28,661 >> loading file tokenizer.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/tokenizer.json
+[INFO|tokenization_utils_base.py:2269] 2025-06-26 23:48:28,661 >> loading file added_tokens.json from cache at None
+[INFO|tokenization_utils_base.py:2269] 2025-06-26 23:48:28,661 >> loading file special_tokens_map.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/special_tokens_map.json
+[INFO|tokenization_utils_base.py:2269] 2025-06-26 23:48:28,661 >> loading file tokenizer_config.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2513] 2025-06-26 23:48:28,919 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
+06/26/2025 23:48:28 - INFO - __main__ - Tokenizer is fast: True
+[INFO|modeling_utils.py:3667] 2025-06-26 23:48:28,923 >> loading weights file model.safetensors from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/model.safetensors.index.json
+[INFO|modeling_utils.py:1591] 2025-06-26 23:48:28,924 >> Instantiating LlamaForCausalLM model under default dtype torch.bfloat16.
+[WARNING|logging.py:328] 2025-06-26 23:48:28,927 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+[INFO|configuration_utils.py:1038] 2025-06-26 23:48:28,928 >> Generate config GenerationConfig {
+  "bos_token_id": 128000,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ]
+}
+
+[WARNING|logging.py:328] 2025-06-26 23:48:28,930 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  4.14it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  3.05it/s][WARNING|logging.py:328] 2025-06-26 23:48:29,418 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  4.72it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  4.71it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  5.66it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  4.86it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  6.37it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.55it/s]
+
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  3.80it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  4.99it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  4.86it/s]
+[INFO|modeling_utils.py:4499] 2025-06-26 23:48:29,827 >> All model checkpoint weights were used when initializing LlamaForCausalLM.
+
+[INFO|modeling_utils.py:4507] 2025-06-26 23:48:29,827 >> All the weights of LlamaForCausalLM were initialized from the model checkpoint at meta-llama/Llama-3.1-8B-Instruct.
+If your task is similar to the task the model of the checkpoint was trained on, you can already use LlamaForCausalLM for predictions without further training.
+[WARNING|logging.py:328] 2025-06-26 23:48:29,863 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  4.70it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]adding special tokens...
+
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  5.82it/s][INFO|configuration_utils.py:993] 2025-06-26 23:48:30,060 >> loading configuration file generation_config.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/generation_config.json
+[INFO|configuration_utils.py:1038] 2025-06-26 23:48:30,060 >> Generate config GenerationConfig {
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "temperature": 0.6,
+  "top_p": 0.9
+}
+
+adding special tokens...
+06/26/2025 23:48:30 - INFO - __main__ - ================ pad, eos, bos, unk, padding ================ 
+06/26/2025 23:48:30 - INFO - __main__ - <|eot_id|>, 128009
+06/26/2025 23:48:30 - INFO - __main__ - <|eot_id|>, 128009
+06/26/2025 23:48:30 - INFO - __main__ - <|begin_of_text|>, 128000
+06/26/2025 23:48:30 - INFO - __main__ - <|reserved_special_token_0|>, 128002
+06/26/2025 23:48:30 - INFO - __main__ - right
+06/26/2025 23:48:30 - INFO - __main__ - lora_r : 8
+06/26/2025 23:48:30 - INFO - __main__ - lora_alpha : 16
+06/26/2025 23:48:30 - INFO - __main__ - lora_dropout : 0.1
+06/26/2025 23:48:30 - INFO - __main__ - lora_target_modules : ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj']
+06/26/2025 23:48:30 - INFO - __main__ - LoRA configs: LoraConfig(task_type='CAUSAL_LM', peft_type=<PeftType.LORA: 'LORA'>, auto_mapping=None, base_model_name_or_path=None, revision=None, inference_mode=False, r=8, target_modules={'v_proj', 'down_proj', 'k_proj', 'up_proj', 'q_proj', 'o_proj', 'gate_proj'}, exclude_modules=None, lora_alpha=16, lora_dropout=0.1, fan_in_fan_out=False, bias='none', use_rslora=False, modules_to_save=None, init_lora_weights=True, layers_to_transform=None, layers_pattern=None, rank_pattern={}, alpha_pattern={}, megatron_config=None, megatron_core='megatron.core', trainable_token_indices=None, loftq_config={}, eva_config=None, corda_config=None, use_dora=False, layer_replication=None, runtime_config=LoraRuntimeConfig(ephemeral_gpu_offload=False), lora_bias=False)
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  6.49it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.79it/s]
+
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  3.82it/s]trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank1]:[W626 23:48:30.009600268 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 1]  using GPU 1 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  4.96it/s]trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+06/26/2025 23:48:30 - INFO - __main__ - block size: 2048
+adding special tokens...
+Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-10eaf7c5c1c6f11a/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-d04c1a71da869374.arrow
+06/26/2025 23:48:30 - INFO - datasets.arrow_dataset - Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-10eaf7c5c1c6f11a/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-d04c1a71da869374.arrow
+
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  5.58it/s]Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-10eaf7c5c1c6f11a/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-930874df0ba118e5.arrow
+06/26/2025 23:48:30 - INFO - datasets.arrow_dataset - Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-10eaf7c5c1c6f11a/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-930874df0ba118e5.arrow
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  6.03it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.56it/s]
+trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank3]:[W626 23:48:30.467276627 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 3]  using GPU 3 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+[rank0]:[W626 23:48:30.494903331 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 0]  using GPU 0 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+adding special tokens...
+trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank2]:[W626 23:48:31.966426045 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 2]  using GPU 2 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+06/26/2025 23:48:32 - INFO - __main__ - xxx: Showcase the tokenized training samples.
+{'input_ids': [128000, 128006, 9125, 128007, 271, 11372, 228, 11372, 103, 87648, 62456, 36278, 237, 11372, 243, 11372, 250, 87648, 36278, 116, 11372, 117, 50228, 107, 11372, 120, 11372, 243, 36278, 245, 11372, 96, 81278, 97, 36278, 114, 81278, 243, 53906, 115, 11372, 243, 60008, 73358, 36278, 255, 28025, 224, 11372, 106, 81278, 243, 50228, 107, 11372, 120, 36278, 228, 11372, 249, 60008, 87648, 100278, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 250, 87648, 53906, 107, 11, 36278, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 248, 81278, 101, 53906, 97, 42412, 36278, 243, 73358, 28025, 223, 87648, 36278, 237, 11372, 105, 11372, 224, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 116, 11372, 106, 50228, 100, 50228, 101, 11372, 253, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 36278, 241, 36278, 116, 11372, 224, 11372, 110, 50228, 103, 11372, 106, 28025, 224, 11372, 110, 11372, 243, 11372, 255, 50228, 105, 60008, 36278, 231, 11372, 103, 11372, 116, 53906, 98, 50228, 103, 87648, 36278, 243, 73358, 28025, 223, 87648, 100278, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 231, 11372, 97, 53906, 97, 73358, 60008, 73358, 36278, 114, 28025, 223, 73358, 28025, 223, 11372, 97, 60008, 330, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 3047, 36278, 110, 81278, 244, 28025, 223, 87648, 11, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 245, 11372, 96, 87648, 50228, 245, 28025, 223, 11372, 110, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 11372, 255, 50228, 105, 60008, 36278, 99, 60008, 11372, 244, 50228, 101, 36278, 237, 11372, 105, 11372, 224, 36278, 114, 60008, 11372, 115, 60008, 36278, 110, 81278, 244, 28025, 223, 87648, 330, 11372, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 28025, 233, 1630, 100278, 498, 36278, 107, 60008, 11372, 244, 50228, 101, 60008, 1630, 36278, 117, 11372, 110, 28025, 233, 36278, 248, 28025, 224, 11372, 94, 11372, 120, 42412, 87648, 53906, 97, 36278, 104, 11372, 110, 50228, 104, 11372, 110, 100278, 36278, 107, 11372, 98, 50228, 108, 53906, 98, 11372, 97, 42412, 36278, 237, 11372, 105, 11372, 224, 36278, 105, 81278, 116, 53906, 97, 50228, 108, 81278, 97, 36278, 105, 53906, 107, 50228, 244, 53906, 107, 50228, 108, 36278, 103, 53906, 108, 11372, 97, 62456, 36278, 107, 11372, 97, 53906, 101, 11372, 114, 28025, 222, 11372, 110, 36278, 98, 50228, 243, 28025, 223, 87648, 100278, 128009, 198, 128006, 882, 128007, 271, 11372, 103, 53906, 108, 11372, 114, 53906, 101, 25, 220, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 400, 17, 11, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 400, 18, 11, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 60008, 73358, 36278, 99, 50228, 106, 400, 87, 36278, 237, 11372, 105, 11372, 224, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 73358, 36278, 99, 50228, 106, 400, 16, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 101, 81278, 250, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 237, 11372, 105, 11372, 224, 36278, 243, 81278, 249, 28025, 223, 36278, 105, 87648, 53906, 100, 28025, 223, 11372, 99, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 243, 81278, 249, 28025, 223, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 100278, 36278, 97, 81278, 101, 62456, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 42412, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 36278, 237, 11372, 105, 11372, 224, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 87648, 100278, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 116, 60008, 36278, 243, 11372, 97, 11372, 253, 42412, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 36278, 243, 73358, 60008, 30, 36278, 228, 11372, 106, 73358, 42412, 36278, 107, 11372, 99, 62456, 36278, 231, 11372, 103, 73358, 60008, 73358, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 231, 11372, 97, 53906, 97, 73358, 220, 18, 36278, 250, 50228, 101, 62456, 36278, 97, 50228, 117, 11372, 110, 60008, 36278, 227, 11372, 250, 50228, 101, 42412, 36278, 248, 11372, 110, 11372, 243, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 243, 11372, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 11372, 228, 11372, 103, 87648, 62456, 36278, 237, 11372, 243, 11372, 250, 87648, 36278, 116, 11372, 117, 50228, 107, 11372, 120, 11372, 243, 36278, 245, 11372, 96, 81278, 97, 36278, 114, 81278, 243, 53906, 115, 11372, 243, 60008, 73358, 36278, 255, 28025, 224, 11372, 106, 81278, 243, 50228, 107, 11372, 120, 36278, 228, 11372, 249, 60008, 87648, 100278, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 250, 87648, 53906, 107, 11, 36278, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 248, 81278, 101, 53906, 97, 42412, 36278, 243, 73358, 28025, 223, 87648, 36278, 237, 11372, 105, 11372, 224, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 116, 11372, 106, 50228, 100, 50228, 101, 11372, 253, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 36278, 241, 36278, 116, 11372, 224, 11372, 110, 50228, 103, 11372, 106, 28025, 224, 11372, 110, 11372, 243, 11372, 255, 50228, 105, 60008, 36278, 231, 11372, 103, 11372, 116, 53906, 98, 50228, 103, 87648, 36278, 243, 73358, 28025, 223, 87648, 100278, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 231, 11372, 97, 53906, 97, 73358, 60008, 73358, 36278, 114, 28025, 223, 73358, 28025, 223, 11372, 97, 60008, 330, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 3047, 36278, 110, 81278, 244, 28025, 223, 87648, 11, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 245, 11372, 96, 87648, 50228, 245, 28025, 223, 11372, 110, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 11372, 255, 50228, 105, 60008, 36278, 99, 60008, 11372, 244, 50228, 101, 36278, 237, 11372, 105, 11372, 224, 36278, 114, 60008, 11372, 115, 60008, 36278, 110, 81278, 244, 28025, 223, 87648, 330, 11372, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 28025, 233, 1630, 100278, 498, 36278, 107, 60008, 11372, 244, 50228, 101, 60008, 1630, 36278, 117, 11372, 110, 28025, 233, 36278, 248, 28025, 224, 11372, 94, 11372, 120, 42412, 87648, 53906, 97, 36278, 104, 11372, 110, 50228, 104, 11372, 110, 100278, 36278, 107, 11372, 98, 50228, 108, 53906, 98, 11372, 97, 42412, 36278, 237, 11372, 105, 11372, 224, 36278, 105, 81278, 116, 53906, 97, 50228, 108, 81278, 97, 36278, 105, 53906, 107, 50228, 244, 53906, 107, 50228, 108, 36278, 103, 53906, 108, 11372, 97, 62456, 36278, 107, 11372, 97, 53906, 101, 11372, 114, 28025, 222, 11372, 110, 36278, 98, 50228, 243, 28025, 223, 87648, 100278, 128009, 198, 128006, 882, 128007, 271, 11372, 103, 53906, 108, 11372, 114, 53906, 101, 25, 220, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 400, 17, 11, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 400, 18, 11, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 60008, 73358, 36278, 99, 50228, 106, 400, 87, 36278, 237, 11372, 105, 11372, 224, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 73358, 36278, 99, 50228, 106, 400, 16, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 101, 81278, 250, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 237, 11372, 105, 11372, 224, 36278, 243, 81278, 249, 28025, 223, 36278, 105, 87648, 53906, 100, 28025, 223, 11372, 99, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 243, 81278, 249, 28025, 223, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 100278, 36278, 97, 81278, 101, 62456, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 42412, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 36278, 237, 11372, 105, 11372, 224, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 87648, 100278, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 116, 60008, 36278, 243, 11372, 97, 11372, 253, 42412, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 36278, 243, 73358, 60008, 30, 36278, 228, 11372, 106, 73358, 42412, 36278, 107, 11372, 99, 62456, 36278, 231, 11372, 103, 73358, 60008, 73358, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 231, 11372, 97, 53906, 97, 73358, 220, 18, 36278, 250, 50228, 101, 62456, 36278, 97, 50228, 117, 11372, 110, 60008, 36278, 227, 11372, 250, 50228, 101, 42412, 36278, 248, 11372, 110, 11372, 243, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 243, 11372, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 11372, 228, 11372, 103, 87648, 62456, 36278, 237, 11372, 243, 11372, 250, 87648, 36278, 116, 11372, 117, 50228, 107, 11372, 120, 11372, 243, 36278, 245, 11372, 96, 81278, 97, 36278, 114, 81278, 243, 53906, 115, 11372, 243, 60008, 73358, 36278, 255, 28025, 224, 11372, 106, 81278, 243, 50228, 107, 11372, 120, 36278, 228, 11372, 249, 60008, 87648, 100278, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 250, 87648, 53906, 107, 11, 36278, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 248, 81278, 101, 53906, 97, 42412, 36278, 243, 73358, 28025, 223, 87648, 36278, 237, 11372, 105, 11372, 224, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 116, 11372, 106, 50228, 100, 50228, 101, 11372, 253, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 36278, 241, 36278, 116, 11372, 224, 11372, 110, 50228, 103, 11372, 106, 28025, 224, 11372, 110, 11372, 243, 11372, 255, 50228, 105, 60008, 36278, 231, 11372, 103, 11372, 116, 53906, 98, 50228, 103, 87648, 36278, 243, 73358, 28025, 223, 87648, 100278, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 231, 11372, 97, 53906, 97, 73358, 60008, 73358, 36278, 114, 28025, 223, 73358, 28025, 223, 11372, 97, 60008, 330, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 3047, 36278, 110, 81278, 244, 28025, 223, 87648, 11, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 245, 11372, 96, 87648, 50228, 245, 28025, 223, 11372, 110, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 11372, 255, 50228, 105, 60008, 36278, 99, 60008, 11372, 244, 50228, 101, 36278, 237, 11372, 105, 11372, 224, 36278, 114, 60008, 11372, 115, 60008, 36278, 110, 81278, 244, 28025, 223, 87648, 330, 11372, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 28025, 233, 1630, 100278, 498, 36278, 107, 60008, 11372, 244, 50228, 101, 60008, 1630, 36278, 117, 11372, 110, 28025, 233, 36278, 248, 28025, 224, 11372, 94, 11372, 120, 42412, 87648, 53906, 97, 36278, 104, 11372, 110, 50228, 104, 11372, 110, 100278, 36278, 107, 11372, 98, 50228, 108, 53906, 98, 11372, 97, 42412, 36278, 237, 11372, 105, 11372, 224, 36278, 105, 81278, 116, 53906, 97, 50228, 108, 81278, 97, 36278, 105, 53906, 107, 50228, 244, 53906, 107, 50228, 108, 36278, 103, 53906, 108, 11372, 97, 62456, 36278, 107, 11372, 97, 53906, 101, 11372, 114, 28025, 222, 11372, 110, 36278, 98, 50228, 243, 28025, 223, 87648, 100278, 128009, 198, 128006, 882, 128007, 271, 11372, 103, 53906, 108, 11372, 114, 53906, 101, 25, 220, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 400, 17, 11, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 400, 18, 11, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 60008, 73358, 36278, 99, 50228, 106, 400, 87, 36278, 237, 11372, 105, 11372, 224, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 73358, 36278, 99, 50228, 106, 400, 16, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 101, 81278, 250, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 237, 11372, 105, 11372, 224, 36278, 243, 81278, 249, 28025, 223, 36278, 105, 87648, 53906, 100, 28025, 223, 11372, 99, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 243, 81278, 249, 28025, 223, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 100278, 36278, 97, 81278, 101, 62456, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 42412, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 36278, 237, 11372, 105, 11372, 224, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 87648, 100278, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 116, 60008, 36278, 243, 11372, 97, 11372, 253, 42412, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 36278, 243, 73358, 60008, 30, 36278, 228, 11372, 106, 73358, 42412, 36278, 107, 11372, 99, 62456, 36278, 231, 11372, 103, 73358, 60008, 73358, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 231, 11372, 97, 53906, 97, 73358, 220, 18, 36278, 250, 50228, 101, 62456, 36278, 97, 50228, 117, 11372, 110, 60008, 36278, 227, 11372, 250, 50228, 101, 42412, 36278, 248, 11372, 110, 11372, 243, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 243, 11372, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 11372, 228, 11372, 103, 87648, 62456, 36278, 237, 11372, 243, 11372, 250, 87648, 36278, 116, 11372, 117, 50228, 107, 11372, 120, 11372, 243, 36278, 245, 11372, 96, 81278, 97, 36278, 114, 81278, 243, 53906, 115, 11372, 243, 60008, 73358, 36278, 255, 28025, 224, 11372, 106, 81278, 243, 50228, 107, 11372, 120, 36278, 228, 11372, 249, 60008, 87648, 100278, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 250, 87648, 53906, 107, 11, 36278, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 248, 81278, 101, 53906, 97, 42412, 36278, 243, 73358, 28025, 223, 87648, 36278, 237, 11372, 105, 11372, 224, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 116, 11372, 106, 50228, 100, 50228, 101, 11372, 253, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 36278, 241, 36278, 116, 11372, 224, 11372, 110, 50228, 103, 11372, 106, 28025, 224, 11372, 110, 11372, 243, 11372, 255, 50228, 105, 60008, 36278, 231, 11372, 103, 11372, 116, 53906, 98, 50228, 103, 87648, 36278, 243, 73358, 28025, 223, 87648, 100278, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 231, 11372, 97, 53906, 97, 73358, 60008, 73358, 36278, 114, 28025, 223, 73358, 28025, 223, 11372, 97, 60008, 330, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 3047, 36278, 110, 81278, 244, 28025, 223, 87648, 11, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 245, 11372, 96, 87648, 50228, 245, 28025, 223, 11372, 110, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 11372, 255, 50228, 105, 60008, 36278, 99, 60008, 11372, 244, 50228, 101, 36278, 237, 11372, 105, 11372, 224, 36278, 114, 60008, 11372, 115, 60008, 36278, 110, 81278, 244, 28025, 223, 87648, 330, 11372, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 28025, 233, 1630, 100278, 498, 36278, 107, 60008, 11372, 244, 50228, 101, 60008, 1630, 36278, 117, 11372, 110, 28025, 233, 36278, 248, 28025, 224, 11372, 94, 11372, 120, 42412, 87648, 53906, 97, 36278, 104, 11372, 110, 50228, 104, 11372, 110, 100278, 36278, 107, 11372, 98, 50228, 108, 53906, 98, 11372, 97, 42412, 36278, 237, 11372, 105, 11372, 224, 36278, 105, 81278, 116, 53906, 97, 50228, 108, 81278, 97, 36278, 105, 53906, 107, 50228, 244, 53906, 107, 50228, 108, 36278, 103, 53906, 108, 11372, 97, 62456, 36278, 107, 11372, 97, 53906, 101, 11372, 114, 28025, 222, 11372, 110, 36278, 98, 50228, 243, 28025, 223, 87648, 100278, 128009, 198, 128006, 882, 128007, 271, 11372, 103, 53906, 108, 11372, 114, 53906, 101, 25, 220, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 400, 17, 11, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 400, 18, 11, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 60008, 73358, 36278, 99, 50228, 106, 400, 87, 36278, 237, 11372, 105, 11372, 224, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 73358, 36278, 99, 50228, 106, 400, 16, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 101, 81278, 250, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 237, 11372, 105, 11372, 224, 36278, 243, 81278, 249, 28025, 223, 36278, 105, 87648, 53906, 100, 28025, 223, 11372, 99, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 243, 81278, 249, 28025, 223, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 100278, 36278, 97, 81278, 101, 62456, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 42412, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 36278, 237, 11372, 105, 11372, 224, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 87648, 100278, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 116, 60008, 36278, 243, 11372, 97, 11372, 253, 42412, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 36278, 243, 73358, 60008, 30, 36278, 228, 11372, 106, 73358, 42412, 36278, 107, 11372, 99, 62456, 36278, 231, 11372, 103, 73358, 60008, 73358, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 231, 11372, 97, 53906, 97, 73358, 220, 18, 36278, 250, 50228, 101, 62456, 36278, 97, 50228, 117, 11372, 110, 60008, 36278, 227, 11372, 250, 50228, 101, 42412, 36278, 248, 11372, 110, 11372, 243, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 243, 11372, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 11372, 228, 11372, 103, 87648, 62456, 36278, 237, 11372, 243, 11372, 250, 87648, 36278, 116, 11372, 117, 50228, 107, 11372, 120, 11372, 243, 36278, 245, 11372, 96, 81278, 97, 36278, 114, 81278, 243, 53906, 115, 11372, 243, 60008, 73358, 36278, 255, 28025, 224, 11372, 106, 81278, 243, 50228, 107, 11372, 120, 36278, 228, 11372, 249, 60008, 87648, 100278, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 250, 87648, 53906, 107, 11, 36278, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 248, 81278, 101, 53906, 97, 42412, 36278, 243, 73358, 28025, 223, 87648, 36278, 237, 11372, 105, 11372, 224, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 116, 11372, 106, 50228, 100, 50228, 101, 11372, 253, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 36278, 241, 36278, 116, 11372, 224, 11372, 110, 50228, 103, 11372, 106, 28025, 224, 11372, 110, 11372, 243, 11372, 255, 50228, 105, 60008, 36278, 231, 11372, 103, 11372, 116, 53906, 98, 50228, 103, 87648, 36278, 243, 73358, 28025, 223, 87648, 100278, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 231, 11372, 97, 53906, 97, 73358, 60008, 73358, 36278, 114, 28025, 223, 73358, 28025, 223, 11372, 97, 60008, 330, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 3047, 36278, 110, 81278, 244, 28025, 223, 87648, 11, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 245, 11372, 96, 87648, 50228, 245, 28025, 223, 11372, 110, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 11372, 255, 50228, 105, 60008, 36278, 99, 60008, 11372, 244, 50228, 101, 36278, 237, 11372, 105, 11372, 224, 36278, 114, 60008, 11372, 115, 60008, 36278, 110, 81278, 244, 28025, 223, 87648, 330, 11372, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 28025, 233, 1630, 100278, 498, 36278, 107, 60008, 11372, 244, 50228, 101, 60008, 1630, 36278, 117, 11372, 110, 28025, 233, 36278, 248, 28025, 224, 11372, 94, 11372, 120, 42412, 87648, 53906, 97, 36278, 104, 11372, 110, 50228, 104, 11372, 110, 100278, 36278, 107, 11372, 98, 50228, 108, 53906, 98, 11372, 97, 42412, 36278, 237, 11372, 105, 11372, 224, 36278, 105, 81278, 116, 53906, 97, 50228, 108, 81278, 97, 36278, 105, 53906, 107, 50228, 244, 53906, 107, 50228, 108, 36278, 103, 53906, 108, 11372, 97, 62456, 36278, 107, 11372, 97, 53906, 101, 11372, 114, 28025, 222, 11372, 110, 36278, 98, 50228, 243, 28025, 223, 87648, 100278, 128009, 198, 128006, 882, 128007, 271, 11372, 103, 53906, 108, 11372, 114, 53906, 101, 25, 220, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 400, 17, 11, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 400, 18, 11, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 60008, 73358, 36278, 99, 50228, 106, 400, 87, 36278, 237, 11372, 105, 11372, 224, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 73358, 36278, 99, 50228, 106, 400, 16, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 101, 81278, 250, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 237, 11372, 105, 11372, 224, 36278, 243, 81278, 249, 28025, 223, 36278, 105, 87648, 53906, 100, 28025, 223, 11372, 99, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 243, 81278, 249, 28025, 223, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 100278, 36278, 97, 81278, 101, 62456, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 42412, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 36278, 237, 11372, 105, 11372, 224, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 87648, 100278, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 116, 60008, 36278, 243, 11372, 97, 11372, 253, 42412, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 36278, 243, 73358, 60008, 30, 36278, 228, 11372, 106, 73358, 42412, 36278, 107, 11372, 99, 62456, 36278, 231, 11372, 103, 73358, 60008, 73358, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 231, 11372, 97, 53906, 97, 73358, 220, 18, 36278, 250, 50228, 101, 62456, 36278, 97, 50228, 117, 11372, 110, 60008, 36278, 227, 11372, 250, 50228, 101, 42412, 36278, 248, 11372, 110, 11372, 243, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 243, 11372, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 11372, 228, 11372, 103, 87648, 62456, 36278, 237, 11372, 243, 11372, 250, 87648, 36278, 116, 11372, 117, 50228, 107, 11372, 120, 11372, 243, 36278, 245, 11372, 96, 81278, 97, 36278, 114, 81278, 243, 53906, 115, 11372, 243, 60008, 73358, 36278, 255, 28025, 224, 11372, 106, 81278, 243, 50228, 107, 11372, 120, 36278, 228, 11372, 249, 60008, 87648, 100278, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 250, 87648, 53906, 107, 11, 36278, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 248, 81278, 101, 53906, 97, 42412, 36278, 243, 73358, 28025, 223, 87648, 36278, 237, 11372, 105, 11372, 224, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 116, 11372, 106, 50228, 100, 50228, 101, 11372, 253, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 36278, 241, 36278, 116, 11372, 224, 11372, 110, 50228, 103, 11372, 106, 28025, 224, 11372, 110, 11372, 243, 11372, 255, 50228, 105, 60008, 36278, 231, 11372, 103, 11372, 116, 53906, 98, 50228, 103, 87648, 36278, 243, 73358, 28025, 223, 87648, 100278, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 231, 11372, 97, 53906, 97, 73358, 60008, 73358, 36278, 114, 28025, 223, 73358, 28025, 223, 11372, 97, 60008, 330, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 3047, 36278, 110, 81278, 244, 28025, 223, 87648, 11, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 245, 11372, 96, 87648, 50228, 245, 28025, 223, 11372, 110, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 11372, 255, 50228, 105, 60008, 36278, 99, 60008, 11372, 244, 50228, 101, 36278, 237, 11372, 105, 11372, 224, 36278, 114, 60008, 11372, 115, 60008, 36278, 110, 81278, 244, 28025, 223, 87648, 330, 11372, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 28025, 233, 1630, 100278, 498, 36278, 107, 60008, 11372, 244, 50228, 101, 60008, 1630, 36278, 117, 11372, 110, 28025, 233, 36278, 248, 28025, 224, 11372, 94, 11372, 120, 42412, 87648, 53906, 97, 36278, 104, 11372, 110, 50228, 104, 11372, 110, 100278, 36278, 107, 11372, 98, 50228, 108, 53906, 98, 11372, 97, 42412, 36278, 237, 11372, 105, 11372, 224, 36278, 105, 81278, 116, 53906, 97, 50228, 108, 81278, 97, 36278, 105, 53906, 107, 50228, 244, 53906, 107, 50228, 108, 36278, 103, 53906, 108, 11372, 97, 62456, 36278, 107, 11372, 97, 53906, 101, 11372, 114, 28025, 222, 11372, 110, 36278, 98, 50228, 243, 28025, 223, 87648, 100278, 128009, 198, 128006, 882, 128007, 271, 11372, 103, 53906, 108, 11372, 114, 53906, 101, 25, 220, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 400, 17, 11, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 400, 18, 11, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 60008, 73358, 36278, 99, 50228, 106, 400, 87, 36278, 237, 11372, 105, 11372, 224, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 73358, 36278, 99, 50228, 106, 400, 16, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 101, 81278, 250, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 237, 11372, 105, 11372, 224, 36278, 243, 81278, 249, 28025, 223, 36278, 105, 87648, 53906, 100, 28025, 223, 11372, 99, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 243, 81278, 249, 28025, 223, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 100278, 36278, 97, 81278, 101, 62456, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 42412, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 36278, 237, 11372, 105, 11372, 224, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 87648, 100278, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 116, 60008, 36278, 243, 11372, 97, 11372, 253, 42412, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 36278, 243, 73358, 60008, 30, 36278, 228, 11372, 106, 73358, 42412, 36278, 107, 11372, 99, 62456, 36278, 231, 11372, 103, 73358, 60008, 73358, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 231, 11372, 97, 53906, 97, 73358, 220, 18, 36278, 250, 50228, 101, 62456, 36278, 97, 50228, 117, 11372, 110, 60008, 36278, 227, 11372, 250, 50228, 101, 42412, 36278, 248, 11372, 110, 11372, 243, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 243, 11372, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 11372, 228, 11372, 103, 87648, 62456, 36278, 237, 11372, 243, 11372, 250, 87648, 36278, 116, 11372, 117, 50228, 107, 11372, 120, 11372, 243, 36278, 245, 11372, 96, 81278, 97, 36278, 114, 81278, 243, 53906, 115, 11372, 243, 60008, 73358, 36278, 255, 28025, 224, 11372, 106, 81278, 243, 50228, 107, 11372, 120, 36278, 228, 11372, 249, 60008, 87648, 100278, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 250, 87648, 53906, 107, 11, 36278, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 248, 81278, 101, 53906, 97, 42412, 36278, 243, 73358, 28025, 223, 87648, 36278, 237, 11372, 105, 11372, 224, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 116, 11372, 106, 50228, 100, 50228, 101, 11372, 253, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 36278, 241, 36278, 116, 11372, 224, 11372, 110, 50228, 103, 11372, 106, 28025, 224, 11372, 110, 11372, 243, 11372, 255, 50228, 105, 60008, 36278, 231, 11372, 103, 11372, 116, 53906, 98, 50228, 103, 87648, 36278, 243, 73358, 28025, 223, 87648, 100278, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 231, 11372, 97, 53906, 97, 73358, 60008, 73358, 36278, 114, 28025, 223, 73358, 28025, 223, 11372, 97, 60008, 330, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 3047, 36278, 110, 81278, 244, 28025, 223, 87648, 11, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 245, 11372, 96, 87648, 50228, 245, 28025, 223, 11372, 110, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 11372, 255, 50228, 105, 60008, 36278, 99, 60008, 11372, 244, 50228, 101, 36278, 237, 11372, 105, 11372, 224, 36278, 114, 60008, 11372, 115, 60008, 36278, 110, 81278, 244, 28025, 223, 87648, 330, 11372, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 28025, 233, 1630, 100278, 498, 36278, 107, 60008, 11372, 244, 50228, 101, 60008, 1630, 36278, 117, 11372, 110, 28025, 233, 36278, 248, 28025, 224, 11372, 94, 11372, 120, 42412, 87648, 53906, 97, 36278, 104, 11372, 110, 50228, 104, 11372, 110, 100278, 36278, 107, 11372, 98, 50228, 108, 53906, 98, 11372, 97, 42412, 36278, 237, 11372, 105, 11372, 224, 36278, 105, 81278, 116, 53906, 97, 50228, 108, 81278, 97, 36278, 105, 53906, 107, 50228, 244, 53906, 107, 50228, 108, 36278, 103, 53906, 108, 11372, 97, 62456, 36278, 107, 11372, 97, 53906, 101, 11372, 114, 28025, 222, 11372, 110, 36278, 98, 50228, 243, 28025, 223, 87648, 100278, 128009, 198, 128006, 882, 128007, 271, 11372, 103, 53906, 108, 11372, 114, 53906, 101, 25, 220, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 400, 17, 11, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 400, 18, 11, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 60008, 73358, 36278, 99, 50228, 106, 400, 87, 36278, 237, 11372, 105, 11372, 224, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 73358, 36278, 99, 50228, 106, 400, 16, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 101, 81278, 250, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 237, 11372, 105, 11372, 224, 36278, 243, 81278, 249, 28025, 223, 36278, 105, 87648, 53906, 100, 28025, 223, 11372, 99, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 243, 81278, 249, 28025, 223, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 100278, 36278, 97, 81278, 101, 62456, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 42412, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 36278, 237, 11372, 105, 11372, 224, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 87648, 100278, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 116, 60008, 36278, 243, 11372, 97, 11372, 253, 42412, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 36278, 243, 73358, 60008, 30, 36278, 228, 11372, 106, 73358, 42412, 36278, 107, 11372, 99, 62456, 36278, 231, 11372, 103, 73358, 60008, 73358, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 231, 11372, 97, 53906, 97, 73358, 220, 18, 36278, 250, 50228, 101, 62456, 36278, 97, 50228, 117, 11372, 110, 60008, 36278, 227, 11372, 250, 50228, 101, 42412, 36278, 248, 11372, 110, 11372, 243, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 243, 11372, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 11372, 228, 11372, 103, 87648, 62456, 36278, 237, 11372, 243, 11372, 250, 87648, 36278, 116, 11372, 117, 50228, 107, 11372, 120, 11372, 243, 36278, 245, 11372, 96, 81278, 97, 36278, 114, 81278, 243, 53906, 115, 11372, 243, 60008, 73358, 36278, 255, 28025, 224, 11372, 106, 81278, 243, 50228, 107, 11372, 120, 36278, 228, 11372, 249, 60008, 87648, 100278, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 250, 87648, 53906, 107, 11, 36278, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 248, 81278, 101, 53906, 97, 42412, 36278, 243, 73358, 28025, 223, 87648, 36278, 237, 11372, 105, 11372, 224, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 116, 11372, 106, 50228, 100, 50228, 101, 11372, 253, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 36278, 241, 36278, 116, 11372, 224, 11372, 110, 50228, 103, 11372, 106, 28025, 224, 11372, 110, 11372, 243, 11372, 255, 50228, 105, 60008, 36278, 231, 11372, 103, 11372, 116, 53906, 98, 50228, 103, 87648, 36278, 243, 73358, 28025, 223, 87648, 100278, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 231, 11372, 97, 53906, 97, 73358, 60008, 73358, 36278, 114, 28025, 223, 73358, 28025, 223, 11372, 97, 60008, 330, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 3047, 36278, 110, 81278, 244, 28025, 223, 87648, 11, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 245, 11372, 96, 87648, 50228, 245, 28025, 223, 11372, 110, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 11372, 255, 50228, 105, 60008, 36278, 99, 60008, 11372, 244, 50228, 101, 36278, 237, 11372, 105, 11372, 224, 36278, 114, 60008, 11372, 115, 60008, 36278, 110, 81278, 244, 28025, 223, 87648, 330, 11372, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 28025, 233, 1630, 100278, 498, 36278, 107, 60008, 11372, 244, 50228, 101, 60008, 1630, 36278, 117, 11372, 110, 28025, 233, 36278, 248, 28025, 224, 11372, 94, 11372, 120, 42412, 87648, 53906, 97, 36278, 104, 11372, 110, 50228, 104, 11372, 110, 100278, 36278, 107, 11372, 98, 50228, 108, 53906, 98, 11372, 97, 42412, 36278, 237, 11372, 105, 11372, 224, 36278, 105, 81278, 116, 53906, 97, 50228, 108, 81278, 97, 36278, 105, 53906, 107, 50228, 244, 53906, 107, 50228, 108, 36278, 103, 53906, 108, 11372, 97, 62456, 36278, 107, 11372, 97, 53906, 101, 11372, 114, 28025, 222, 11372, 110, 36278, 98, 50228, 243, 28025, 223, 87648, 100278, 128009, 198, 128006, 882, 128007, 271, 11372, 103, 53906, 108, 11372, 114, 53906, 101, 25, 220, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 400, 17, 11, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 400, 18, 11, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 60008, 73358, 36278, 99, 50228, 106, 400, 87, 36278, 237, 11372, 105, 11372, 224, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 73358, 36278, 99, 50228, 106, 400, 16, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 101, 81278, 250, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 237, 11372, 105, 11372, 224, 36278, 243, 81278, 249, 28025, 223, 36278, 105, 87648, 53906, 100, 28025, 223, 11372, 99, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 243, 81278, 249, 28025, 223, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 100278, 36278, 97, 81278, 101, 62456, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 42412, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 36278, 237, 11372, 105, 11372, 224, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 87648, 100278, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 116, 60008, 36278, 243, 11372, 97, 11372, 253, 42412, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 36278, 243, 73358, 60008, 30, 36278, 228, 11372, 106, 73358, 42412, 36278, 107, 11372, 99, 62456, 36278, 231, 11372, 103, 73358, 60008, 73358, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 231, 11372, 97, 53906, 97, 73358, 220, 18, 36278, 250, 50228, 101, 62456, 36278, 97, 50228, 117, 11372, 110, 60008, 36278, 227, 11372, 250, 50228, 101, 42412, 36278, 248, 11372, 110, 11372, 243, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 243, 11372, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -10{'input_ids': [128000, 128006, 9125, 128007, 271, 11372, 228, 11372, 103, 87648, 62456, 36278, 237, 11372, 243, 11372, 250, 87648, 36278, 116, 11372, 117, 50228, 107, 11372, 120, 11372, 243, 36278, 245, 11372, 96, 81278, 97, 36278, 114, 81278, 243, 53906, 115, 11372, 243, 60008, 73358, 36278, 255, 28025, 224, 11372, 106, 81278, 243, 50228, 107, 11372, 120, 36278, 228, 11372, 249, 60008, 87648, 100278, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 250, 87648, 53906, 107, 11, 36278, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 248, 81278, 101, 53906, 97, 42412, 36278, 243, 73358, 28025, 223, 87648, 36278, 237, 11372, 105, 11372, 224, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 116, 11372, 106, 50228, 100, 50228, 101, 11372, 253, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 36278, 241, 36278, 116, 11372, 224, 11372, 110, 50228, 103, 11372, 106, 28025, 224, 11372, 110, 11372, 243, 11372, 255, 50228, 105, 60008, 36278, 231, 11372, 103, 11372, 116, 53906, 98, 50228, 103, 87648, 36278, 243, 73358, 28025, 223, 87648, 100278, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 231, 11372, 97, 53906, 97, 73358, 60008, 73358, 36278, 114, 28025, 223, 73358, 28025, 223, 11372, 97, 60008, 330, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 3047, 36278, 110, 81278, 244, 28025, 223, 87648, 11, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 245, 11372, 96, 87648, 50228, 245, 28025, 223, 11372, 110, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 11372, 255, 50228, 105, 60008, 36278, 99, 60008, 11372, 244, 50228, 101, 36278, 237, 11372, 105, 11372, 224, 36278, 114, 60008, 11372, 115, 60008, 36278, 110, 81278, 244, 28025, 223, 87648, 330, 11372, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 28025, 233, 1630, 100278, 498, 36278, 107, 60008, 11372, 244, 50228, 101, 60008, 1630, 36278, 117, 11372, 110, 28025, 233, 36278, 248, 28025, 224, 11372, 94, 11372, 120, 42412, 87648, 53906, 97, 36278, 104, 11372, 110, 50228, 104, 11372, 110, 100278, 36278, 107, 11372, 98, 50228, 108, 53906, 98, 11372, 97, 42412, 36278, 237, 11372, 105, 11372, 224, 36278, 105, 81278, 116, 53906, 97, 50228, 108, 81278, 97, 36278, 105, 53906, 107, 50228, 244, 53906, 107, 50228, 108, 36278, 103, 53906, 108, 11372, 97, 62456, 36278, 107, 11372, 97, 53906, 101, 11372, 114, 28025, 222, 11372, 110, 36278, 98, 50228, 243, 28025, 223, 87648, 100278, 128009, 198, 128006, 882, 128007, 271, 11372, 103, 53906, 108, 11372, 114, 53906, 101, 25, 220, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 400, 17, 11, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 400, 18, 11, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 60008, 73358, 36278, 99, 50228, 106, 400, 87, 36278, 237, 11372, 105, 11372, 224, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 73358, 36278, 99, 50228, 106, 400, 16, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 101, 81278, 250, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 237, 11372, 105, 11372, 224, 36278, 243, 81278, 249, 28025, 223, 36278, 105, 87648, 53906, 100, 28025, 223, 11372, 99, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 243, 81278, 249, 28025, 223, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 100278, 36278, 97, 81278, 101, 62456, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 42412, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 36278, 237, 11372, 105, 11372, 224, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 87648, 100278, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 116, 60008, 36278, 243, 11372, 97, 11372, 253, 42412, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 36278, 243, 73358, 60008, 30, 36278, 228, 11372, 106, 73358, 42412, 36278, 107, 11372, 99, 62456, 36278, 231, 11372, 103, 73358, 60008, 73358, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 231, 11372, 97, 53906, 97, 73358, 220, 18, 36278, 250, 50228, 101, 62456, 36278, 97, 50228, 117, 11372, 110, 60008, 36278, 227, 11372, 250, 50228, 101, 42412, 36278, 248, 11372, 110, 11372, 243, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 243, 11372, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 28{'input_ids': [128000, 128006, 9125, 128007, 271, 11372, 228, 11372, 103, 87648, 62456, 36278, 237, 11372, 243, 11372, 250, 87648, 36278, 116, 11372, 117, 50228, 107, 11372, 120, 11372, 243, 36278, 245, 11372, 96, 81278, 97, 36278, 114, 81278, 243, 53906, 115, 11372, 243, 60008, 73358, 36278, 255, 28025, 224, 11372, 106, 81278, 243, 50228, 107, 11372, 120, 36278, 228, 11372, 249, 60008, 87648, 100278, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 250, 87648, 53906, 107, 11, 36278, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 248, 81278, 101, 53906, 97, 42412, 36278, 243, 73358, 28025, 223, 87648, 36278, 237, 11372, 105, 11372, 224, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 116, 11372, 106, 50228, 100, 50228, 101, 11372, 253, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 36278, 241, 36278, 116, 11372, 224, 11372, 110, 50228, 103, 11372, 106, 28025, 224, 11372, 110, 11372, 243, 11372, 255, 50228, 105, 60008, 36278, 231, 11372, 103, 11372, 116, 53906, 98, 50228, 103, 87648, 36278, 243, 73358, 28025, 223, 87648, 100278, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 231, 11372, 97, 53906, 97, 73358, 60008, 73358, 36278, 114, 28025, 223, 73358, 28025, 223, 11372, 97, 60008, 330, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 3047, 36278, 110, 81278, 244, 28025, 223, 87648, 11, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 245, 11372, 96, 87648, 50228, 245, 28025, 223, 11372, 110, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 11372, 255, 50228, 105, 60008, 36278, 99, 60008, 11372, 244, 50228, 101, 36278, 237, 11372, 105, 11372, 224, 36278, 114, 60008, 11372, 115, 60008, 36278, 110, 81278, 244, 28025, 223, 87648, 330, 11372, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 28025, 233, 1630, 100278, 498, 36278, 107, 60008, 11372, 244, 50228, 101, 60008, 1630, 36278, 117, 11372, 110, 28025, 233, 36278, 248, 28025, 224, 11372, 94, 11372, 120, 42412, 87648, 53906, 97, 36278, 104, 11372, 110, 50228, 104, 11372, 110, 100278, 36278, 107, 11372, 98, 50228, 108, 53906, 98, 11372, 97, 42412, 36278, 237, 11372, 105, 11372, 224, 36278, 105, 81278, 116, 53906, 97, 50228, 108, 81278, 97, 36278, 105, 53906, 107, 50228, 244, 53906, 107, 50228, 108, 36278, 103, 53906, 108, 11372, 97, 62456, 36278, 107, 11372, 97, 53906, 101, 11372, 114, 28025, 222, 11372, 110, 36278, 98, 50228, 243, 28025, 223, 87648, 100278, 128009, 198, 128006, 882, 128007, 271, 11372, 103, 53906, 108, 11372, 114, 53906, 101, 25, 220, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 400, 17, 11, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 400, 18, 11, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 60008, 73358, 36278, 99, 50228, 106, 400, 87, 36278, 237, 11372, 105, 11372, 224, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 73358, 36278, 99, 50228, 106, 400, 16, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 101, 81278, 250, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 237, 11372, 105, 11372, 224, 36278, 243, 81278, 249, 28025, 223, 36278, 105, 87648, 53906, 100, 28025, 223, 11372, 99, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 243, 81278, 249, 28025, 223, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 100278, 36278, 97, 81278, 101, 62456, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 42412, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 36278, 237, 11372, 105, 11372, 224, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 87648, 100278, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 116, 60008, 36278, 243, 11372, 97, 11372, 253, 42412, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 36278, 243, 73358, 60008, 30, 36278, 228, 11372, 106, 73358, 42412, 36278, 107, 11372, 99, 62456, 36278, 231, 11372, 103, 73358, 60008, 73358, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 231, 11372, 97, 53906, 97, 73358, 220, 18, 36278, 250, 50228, 101, 62456, 36278, 97, 50228, 117, 11372, 110, 60008, 36278, 227, 11372, 250, 50228, 101, 42412, 36278, 248, 11372, 110, 11372, 243, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 243, 11372, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 280, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009]}4, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -4, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009]}
+
+100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 11372, 228, 11372, 103, 87648, 62456, 36278, 237, 11372, 243, 11372, 250, 87648, 36278, 116, 11372, 117, 50228, 107, 11372, 120, 11372, 243, 36278, 245, 11372, 96, 81278, 97, 36278, 114, 81278, 243, 53906, 115, 11372, 243, 60008, 73358, 36278, 255, 28025, 224, 11372, 106, 81278, 243, 50228, 107, 11372, 120, 36278, 228, 11372, 249, 60008, 87648, 100278, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 250, 87648, 53906, 107, 11, 36278, 100{'input_ids': [128000, 128006, 9125, 128007, 271, 11372, 228, 11372, 103, 87648, 62456, 36278, 237, 11372, 243, 11372, 250, 87648, 36278, 116, 11372, 117, 50228, 107, 11372, 120, 11372, 243, 36278, 245, 11372, 96, 81278, 97, 36278, 114, 81278, 243, 53906, 115, 11372, 243, 60008, 73358, 36278, 255, 28025, 224, 11372, 106, 81278, 243, 50228, 107, 11372, 120, 36278, 228, 11372, 249, 60008, 87648, 100278, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 250, 87648, 53906, 107, 11, 36278, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 248, 81278, 101, 53906, 97, 42412, 36278, 243, 73358, 28025, 223, 87648, 36278, 237, 11372, 105, 11372, 224, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 116, 11372, 106, 50228, 100, 50228, 101, 11372, 253, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 36278, 241, 36278, 116, 11372, 224, 11372, 110, 50228, 103, 11372, 106, 28025, 224, 11372, 110, 11372, 243, 11372, 255, 50228, 105, 60008, 36278, 231, 11372, 103, 11372, 116, 53906, 98, 50228, 103, 87648, 36278, 243, 73358, 28025, 223, 87648, 100278, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 231, 11372, 97, 53906, 97, 73358, 60008, 73358, 36278, 114, 28025, 223, 73358, 28025, 223, 11372, 97, 60008, 330, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 3047, 36278, 110, 81278, 244, 28025, 223, 87648, 11, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 245, 11372, 96, 87648, 50228, 245, 28025, 223, 11372, 110, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 11372, 255, 50228, 105, 60008, 36278, 99, 60008, 11372, 244, 50228, 101, 36278, 237, 11372, 105, 11372, 224, 36278, 114, 60008, 11372, 115, 60008, 36278, 110, 81278, 244, 28025, 223, 87648, 330, 11372, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 28025, 233, 1630, 100278, 498, 36278, 107, 60008, 11372, 244, 50228, 101, 60008, 1630, 36278, 117, 11372, 110, 28025, 233, 36278, 248, 28025, 224, 11372, 94, 11372, 120, 42412, 87648, 53906, 97, 36278, 104, 11372, 110, 50228, 104, 11372, 110, 100278, 36278, 107, 11372, 98, 50228, 108, 53906, 98, 11372, 97, 42412, 36278, 237, 11372, 105, 11372, 224, 36278, 105, 81278, 116, 53906, 97, 50228, 108, 81278, 97, 36278, 105, 53906, 107, 50228, 244, 53906, 107, 50228, 108, 36278, 103, 53906, 108, 11372, 97, 62456, 36278, 107, 11372, 97, 53906, 101, 11372, 114, 28025, 222, 11372, 110, 36278, 98, 50228, 243, 28025, 223, 87648, 100278, 128009, 198, 128006, 882, 128007, 271, 11372, 103, 53906, 108, 11372, 114, 53906, 101, 25, 220, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 400, 17, 11, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 400, 18, 11, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 60008, 73358, 36278, 99, 50228, 106, 400, 87, 36278, 237, 11372, 105, 11372, 224, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 73358, 36278, 99, 50228, 106, 400, 16, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 101, 81278, 250, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 237, 11372, 105, 11372, 224, 36278, 243, 81278, 249, 28025, 223, 36278, 105, 87648, 53906, 100, 28025, 223, 11372, 99, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 243, 81278, 249, 28025, 223, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 100278, 36278, 97, 81278, 101, 62456, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 42412, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 36278, 237, 11372, 105, 11372, 224, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 87648, 100278, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 116, 60008, 36278, 243, 11372, 97, 11372, 253, 42412, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 36278, 243, 73358, 60008, 30, 36278, 228, 11372, 106, 73358, 42412, 36278, 107, 11372, 99, 62456, 36278, 231, 11372, 103, 73358, 60008, 73358, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 231, 11372, 97, 53906, 97, 73358, 220, 18, 36278, 250, 50228, 101, 62456, 36278, 97, 50228, 117, 11372, 110, 60008, 36278, 227, 11372, 250, 50228, 101, 42412, 36278, 248, 11372, 110, 11372, 243, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 243, 11372, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 248, 81278, 101, 53906, 97, 42412, 36278, 243, 73358, 28025, 223, 87648, 36278, 237, 11372, 105, 11372, 224, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 116, 11372, 106, 50228, 100, 50228, 101, 11372, 253, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 36278, 241, 36278, 116, 11372, 224, 11372, 110, 50228, 103, 11372, 106, 28025, 224, 11372, 110, 11372, 243, 11372, 255, 50228, 105, 60008, 36278, 231, 11372, 103, 11372, 116, 53906, 98, 50228, 103, 87648, 36278, 243, 73358, 28025, 223, 87648, 100278, 36278, 228, 11372, 103, 87648, 50228, 108, 36278, 231, 11372, 97, 53906, 97, 73358, 60008, 73358, 36278, 114, 28025, 223, 73358, 28025, 223, 11372, 97, 60008, 330, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 3047, 36278, 110, 81278, 244, 28025, 223, 87648, 11, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 245, 11372, 96, 87648, 50228, 245, 28025, 223, 11372, 110, 62456, 36278, 116, 53906, 103, 11372, 115, 53906, 253, 11372, 255, 50228, 105, 60008, 36278, 99, 60008, 11372, 244, 50228, 101, 36278, 237, 11372, 105, 11372, 224, 36278, 114, 60008, 11372, 115, 60008, 36278, 110, 81278, 244, 28025, 223, 87648, 330, 11372, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 28025, 233, 1630, 100278, 498, 36278, 107, 60008, 11372, 244, 50228, 101, 60008, 1630, 36278, 117, 11372, 110, 28025, 233, 36278, 248, 28025, 224, 11372, 94, 11372, 120, 42412, 87648, 53906, 97, 36278, 104, 11372, 110, 50228, 104, 11372, 110, 100278, 36278, 107, 11372, 98, 50228, 108, 53906, 98, 11372, 97, 42412, 36278, 237, 11372, 105, 11372, 224, 36278, 105, 81278, 116, 53906, 97, 50228, 108, 81278, 97, 36278, 105, 53906, 107, 50228, 244, 53906, 107, 50228, 108, 36278, 103, 53906, 108, 11372, 97, 62456, 36278, 107, 11372, 97, 53906, 101, 11372, 114, 28025, 222, 11372, 110, 36278, 98, 50228, 243, 28025, 223, 87648, 100278, 128009, 198, 128006, 882, 128007, 271, 11372, 103, 53906, 108, 11372, 114, 53906, 101, 25, 220, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 400, 17, 11, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 400, 18, 11, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 60008, 73358, 36278, 99, 50228, 106, 400, 87, 36278, 237, 11372, 105, 11372, 224, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 103, 53906, 108, 11372, 97, 81278, 253, 62456, 73358, 36278, 99, 50228, 106, 400, 16, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 101, 81278, 250, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 237, 11372, 105, 11372, 224, 36278, 243, 81278, 249, 28025, 223, 36278, 105, 87648, 53906, 100, 28025, 223, 11372, 99, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 243, 81278, 249, 28025, 223, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 100278, 36278, 97, 81278, 101, 62456, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 42412, 11, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 11372, 116, 36278, 237, 11372, 105, 11372, 224, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 36278, 227, 73358, 53906, 94, 50228, 108, 36278, 243, 73358, 60008, 87648, 100278, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 60008, 73358, 36278, 250, 87648, 53906, 107, 36278, 116, 60008, 36278, 243, 11372, 97, 11372, 253, 42412, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 36278, 243, 73358, 60008, 30, 36278, 228, 11372, 106, 73358, 42412, 36278, 107, 11372, 99, 62456, 36278, 231, 11372, 103, 73358, 60008, 73358, 36278, 103, 53906, 108, 11372, 114, 53906, 101, 60008, 73358, 36278, 231, 11372, 97, 53906, 97, 73358, 220, 18, 36278, 250, 50228, 101, 62456, 36278, 97, 50228, 117, 11372, 110, 60008, 36278, 227, 11372, 250, 50228, 101, 42412, 36278, 248, 11372, 110, 11372, 243, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 243, 11372, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -10, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009]}
+0, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 11372, 100, 50228, 103, 60008, 36278, 100, 50228, 103, 60008, 36278, 231, 11372, 97, 53906, 97, 73358, 25, 36278, 97, 81278, 101, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 103, 28025, 223, 11372, 248, 81278, 101, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 18, 353, 400, 17, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 228, 11372, 229, 11372, 116, 11372, 94, 36278, 248, 50228, 107, 11372, 120, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 18, 284, 400, 21, 100278, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 243, 53906, 107, 50228, 104, 60008, 36278, 110, 53906, 107, 50228, 253, 60008, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 87, 284, 400, 17, 87, 28025, 115, 36278, 99, 28025, 223, 11372, 253, 62456, 36278, 237, 11372, 116, 11372, 103, 53906, 108, 60008, 11372, 116, 28025, 233, 73358, 36278, 99, 50228, 106, 220, 17, 353, 400, 16, 284, 400, 17, 100278, 36278, 116, 11372, 106, 11372, 116, 53906, 97, 36278, 103, 50228, 101, 28025, 222, 11372, 107, 11372, 120, 60008, 73358, 36278, 106, 28025, 233, 11372, 253, 36278, 106, 28025, 224, 11372, 110, 53906, 107, 36278, 117, 11372, 110, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 100278, 36278, 116, 53906, 107, 50228, 101, 53906, 94, 62456, 36278, 105, 81278, 114, 36278, 94, 11372, 110, 50228, 108, 60008, 73358, 36278, 105, 81278, 110, 36278, 99, 81278, 107, 11372, 120, 60008, 36278, 227, 73358, 53906, 98, 36278, 103, 53906, 108, 11372, 99, 50228, 101, 36278, 243, 73358, 60008, 11, 36278, 97, 50228, 229, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 104, 60008, 73358, 11372, 97, 36278, 103, 50228, 107, 11372, 120, 36278, 97, 42412, 36278, 117, 11372, 110, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 28025, 115, 36278, 228, 11372, 106, 50228, 99, 60008, 73358, 36278, 99, 60008, 11372, 241, 11372, 107, 11372, 120, 42412, 36278, 117, 11372, 107, 11372, 120, 60008, 11372, 249, 60008, 36278, 107, 60008, 36278, 116, 60008, 36278, 107, 60008, 36278, 103, 73358, 81278, 105, 73358, 53906, 97, 87648, 11372, 253, 62456, 36278, 103, 50228, 105, 60008, 36278, 97, 42412, 36278, 117, 11372, 110, 220, 18, 36278, 94, 11372, 110, 50228, 108, 11, 36278, 97, 50228, 229, 36278, 228, 11372, 106, 73358, 42412, 36278, 110, 81278, 244, 11372, 97, 60008, 36278, 103, 50228, 108, 62456, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 28025, 115, 36278, 116, 73358, 11372, 110, 28025, 222, 11372, 243, 73358, 11372, 96, 36278, 243, 73358, 60008, 11, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 400, 17, 87, 284, 400, 18, 100278, 36278, 231, 11372, 255, 11372, 107, 11372, 120, 36278, 103, 11372, 243, 53906, 115, 11372, 243, 60008, 400, 17, 36278, 99, 53906, 105, 50228, 108, 42412, 36278, 255, 50228, 245, 36278, 243, 73358, 11372, 110, 60008, 36278, 228, 11372, 106, 73358, 42412, 36278, 103, 50228, 229, 25, 865, 284, 400, 16, 13, 1135, 100278, 865, 36278, 237, 73358, 36278, 106, 50228, 101, 36278, 117, 11372, 110, 400, 16, 13, 1135, 100278, 36278, 231, 11372, 97, 53906, 97, 73358, 36278, 117, 11372, 110, 220, 16, 13, 20, 100278, 128009]}
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+[INFO|trainer.py:658] 2025-06-26 23:48:34,726 >> Using auto half precision backend
+[2025-06-26 23:48:34,996] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed info: version=0.17.1, git-hash=unknown, git-branch=unknown
+[2025-06-26 23:48:34,996] [INFO] [config.py:655:__init__] Config mesh_device None world_size = 4
+[2025-06-26 23:48:39,308] [INFO] [engine.py:1325:_configure_distributed_model] ********** distributed groups summary **********
+	 self.dp_world_size=4
+	 self.mp_world_size=1
+	 self.seq_dp_world_size=4
+	 self.sequence_parallel_size=1
+***********************************************
+[2025-06-26 23:48:39,902] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Detected CUDA files, patching ldflags
+Emitting ninja build file /home/iitm_admin/.cache/torch_extensions/py39_cu126/cpu_adam/build.ninja...
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/utils/cpp_extension.py:2356: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
+If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
+  warnings.warn(
+Building extension module cpu_adam...
+Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+ninja: no work to do.
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 2.7387800216674805 seconds
+Adam Optimizer #0 is created with AVX512 arithmetic capability.
+Config: alpha=0.000500, betas=(0.900000, 0.999000), weight_decay=0.000000, adam_w=1
+[2025-06-26 23:48:44,061] [INFO] [logging.py:107:log_dist] [Rank 0] Using DeepSpeed Optimizer param name adam as basic optimizer
+[2025-06-26 23:48:44,061] [INFO] [logging.py:107:log_dist] [Rank 0] Removing param_group that has no 'params' in the basic Optimizer
+[2025-06-26 23:48:44,109] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Basic Optimizer = DeepSpeedCPUAdam
+[2025-06-26 23:48:44,110] [INFO] [utils.py:59:is_zero_supported_optimizer] Checking ZeRO support for optimizer=DeepSpeedCPUAdam type=<class 'deepspeed.ops.adam.cpu_adam.DeepSpeedCPUAdam'>
+[2025-06-26 23:48:44,110] [INFO] [logging.py:107:log_dist] [Rank 0] Creating torch.bfloat16 ZeRO stage 1 optimizer
+[2025-06-26 23:48:44,110] [INFO] [stage_1_and_2.py:151:__init__] Reduce bucket size 200000000
+[2025-06-26 23:48:44,110] [INFO] [stage_1_and_2.py:152:__init__] Allgather bucket size 200000000
+[2025-06-26 23:48:44,110] [INFO] [stage_1_and_2.py:153:__init__] CPU Offload: True
+[2025-06-26 23:48:44,110] [INFO] [stage_1_and_2.py:154:__init__] Round robin gradient partitioning: False
+Loading extension module cpu_adam...
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 2.7846410274505615 seconds
+Time to load cpu_adam op: 2.8138020038604736 seconds
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 2.7922706604003906 seconds
+[2025-06-26 23:48:44,515] [INFO] [utils.py:781:see_memory_usage] Before initializing optimizer states
+[2025-06-26 23:48:44,515] [INFO] [utils.py:782:see_memory_usage] MA 15.0 GB         Max_MA 15.0 GB         CA 15.16 GB         Max_CA 15 GB 
+[2025-06-26 23:48:44,516] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 77.31 GB, percent = 3.8%
+[2025-06-26 23:48:44,744] [INFO] [utils.py:781:see_memory_usage] After initializing optimizer states
+[2025-06-26 23:48:44,745] [INFO] [utils.py:782:see_memory_usage] MA 15.0 GB         Max_MA 15.0 GB         CA 15.16 GB         Max_CA 15 GB 
+[2025-06-26 23:48:44,745] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 77.44 GB, percent = 3.8%
+[2025-06-26 23:48:44,745] [INFO] [stage_1_and_2.py:573:__init__] optimizer state initialized
+[2025-06-26 23:48:44,906] [INFO] [utils.py:781:see_memory_usage] After initializing ZeRO optimizer
+[2025-06-26 23:48:44,907] [INFO] [utils.py:782:see_memory_usage] MA 15.0 GB         Max_MA 15.0 GB         CA 15.16 GB         Max_CA 15 GB 
+[2025-06-26 23:48:44,907] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 77.52 GB, percent = 3.8%
+[2025-06-26 23:48:44,910] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Final Optimizer = DeepSpeedZeroOptimizer
+[2025-06-26 23:48:44,910] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed using client callable to create LR scheduler
+[2025-06-26 23:48:44,910] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed LR Scheduler = <torch.optim.lr_scheduler.LambdaLR object at 0x768cc22828b0>
+[2025-06-26 23:48:44,910] [INFO] [logging.py:107:log_dist] [Rank 0] step=0, skipped=0, lr=[0.0], mom=[[0.9, 0.999]]
+[2025-06-26 23:48:44,916] [INFO] [logging.py:107:log_dist] [Rank 0] [TorchCheckpointEngine] Initialized with serialization = True
+[2025-06-26 23:48:44,916] [INFO] [config.py:921:print] DeepSpeedEngine configuration:
+[2025-06-26 23:48:44,916] [INFO] [config.py:925:print]   activation_checkpointing_config  {
+    "partition_activations": false, 
+    "contiguous_memory_optimization": false, 
+    "cpu_checkpointing": false, 
+    "number_checkpoints": null, 
+    "synchronize_checkpoint_boundary": false, 
+    "profile": false
+}
+[2025-06-26 23:48:44,916] [INFO] [config.py:925:print]   aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'intra_op_parallelism': 1, 'single_submit': False, 'overlap_events': True, 'use_gds': False}
+[2025-06-26 23:48:44,916] [INFO] [config.py:925:print]   amp_enabled .................. False
+[2025-06-26 23:48:44,916] [INFO] [config.py:925:print]   amp_params ................... False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   autotuning_config ............ {
+    "enabled": false, 
+    "start_step": null, 
+    "end_step": null, 
+    "metric_path": null, 
+    "arg_mappings": null, 
+    "metric": "throughput", 
+    "model_info": null, 
+    "results_dir": "autotuning_results", 
+    "exps_dir": "autotuning_exps", 
+    "overwrite": true, 
+    "fast": true, 
+    "start_profile_step": 3, 
+    "end_profile_step": 5, 
+    "tuner_type": "gridsearch", 
+    "tuner_early_stopping": 5, 
+    "tuner_num_trials": 50, 
+    "model_info_path": null, 
+    "mp_size": 1, 
+    "max_train_batch_size": null, 
+    "min_train_batch_size": 1, 
+    "max_train_micro_batch_size_per_gpu": 1.024000e+03, 
+    "min_train_micro_batch_size_per_gpu": 1, 
+    "num_tuning_micro_batch_sizes": 3
+}
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   bfloat16_config .............. enabled=True immediate_grad_update=False check_grad_overflow=False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   checkpoint_config ............ {'tag_validation': 'WARN', 'checkpoint_serialization': True, 'writer': None}
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   checkpoint_parallel_write_pipeline  False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   checkpoint_tag_validation_enabled  True
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   checkpoint_tag_validation_fail  False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   comms_config ................. <deepspeed.comm.config.DeepSpeedCommsConfig object at 0x768cc2825ca0>
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   communication_data_type ...... None
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   compile_config ............... deepcompile=False free_activation=False offload_activation=False offload_opt_states=False double_buffer=True symmetric_memory=False debug_log=False offload_parameters=False sync_before_reduce=False sync_after_reduce=False sync_before_allgather=False sync_after_allgather=False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}}
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   curriculum_enabled_legacy .... False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   curriculum_params_legacy ..... False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'pin_memory': False, 'curriculum_learning': {'enabled': False}, 'dynamic_batching': {'enabled': False, 'lr_scaling_method': 'linear', 'min_batch_size': 1, 'max_batch_size': None, 'sequence_picking_order': 'dataloader', 'verbose': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}}
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   data_efficiency_enabled ...... False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   dataloader_drop_last ......... False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   disable_allgather ............ False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   dump_state ................... False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   eigenvalue_enabled ........... False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   eigenvalue_gas_boundary_resolution  1
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   eigenvalue_layer_name ........ bert.encoder.layer
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   eigenvalue_layer_num ......... 0
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   eigenvalue_max_iter .......... 100
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   eigenvalue_stability ......... 1e-06
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   eigenvalue_tol ............... 0.01
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   eigenvalue_verbose ........... False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   elasticity_enabled ........... False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   float16_config ............... enabled=False auto_cast=False loss_scale=0.0 initial_scale_power=16 loss_scale_window=1000 hysteresis=2 consecutive_hysteresis=False min_loss_scale=1 fp16_master_weights_and_grads=False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   flops_profiler_config ........ {
+    "enabled": false, 
+    "recompute_fwd_factor": 0.0, 
+    "profile_step": 1, 
+    "module_depth": -1, 
+    "top_modules": 1, 
+    "detailed": true, 
+    "output_file": null
+}
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   global_rank .................. 0
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   grad_accum_dtype ............. None
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   gradient_accumulation_steps .. 1
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   gradient_clipping ............ 1.0
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   gradient_predivide_factor .... 1.0
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   graph_harvesting ............. False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   load_universal_checkpoint .... False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   memory_breakdown ............. False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   mics_hierarchial_params_gather  False
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   mics_shard_size .............. -1
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') comet=CometConfig(enabled=False, samples_log_interval=100, project=None, workspace=None, api_key=None, experiment_name=None, experiment_key=None, online=None, mode=None) wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName')
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   nebula_config ................ {
+    "enabled": false, 
+    "persistent_storage_path": null, 
+    "persistent_time_interval": 100, 
+    "num_of_version_in_retention": 2, 
+    "enable_nebula_load": true, 
+    "load_path": null
+}
+[2025-06-26 23:48:44,917] [INFO] [config.py:925:print]   optimizer_legacy_fusion ...... False
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   optimizer_name ............... adam
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   optimizer_params ............. {'lr': 0.0005, 'betas': [0.9, 0.999], 'eps': 1e-08, 'weight_decay': 0.0}
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0, 'pipe_partitioned': True, 'grad_partitioned': True}
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   pld_enabled .................. False
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   pld_params ................... False
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   prescale_gradients ........... False
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   scheduler_name ............... None
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   scheduler_params ............. None
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   seq_parallel_communication_data_type  torch.float32
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   sparse_attention ............. None
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   sparse_gradients_enabled ..... False
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   steps_per_print .............. inf
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   tensor_parallel_config ....... dtype=torch.float16 autotp_size=0 tp_overlap_comm=False tensor_parallel=TPConfig(tp_size=1, tp_grain_size=1, mpu=None, tp_group=None) injection_policy_tuple=None keep_module_on_host=False replace_with_kernel_inject=False
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   timers_config ................ enabled=True synchronized=True
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   train_batch_size ............. 100
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   train_micro_batch_size_per_gpu  25
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   use_data_before_expert_parallel_  False
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   use_node_local_storage ....... False
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   wall_clock_breakdown ......... False
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   weight_quantization_config ... None
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   world_size ................... 4
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   zero_allow_untested_optimizer  False
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   zero_config .................. stage=1 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=200000000 use_multi_rank_bucket_allreduce=True allgather_partitions=True allgather_bucket_size=200000000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=None offload_optimizer=DeepSpeedZeroOffloadOptimizerConfig(device='cpu', nvme_path=None, buffer_count=4, pin_memory=True, pipeline_read=False, pipeline_write=False, fast_init=False, ratio=1.0) sub_group_size=1000000000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50000000 param_persistence_threshold=100000 model_persistence_threshold=9223372036854775807 max_live_parameters=1000000000 max_reuse_distance=1000000000 gather_16bit_weights_on_model_save=False module_granularity_threshold=0 use_all_reduce_for_fetch_params=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False zero_hpz_partition_size=1 zero_quantized_weights=False zero_quantized_nontrainable_weights=False zero_quantized_gradients=False zeropp_loco_param=None mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=True pipeline_loading_checkpoint=False override_module_apply=True log_trace_cache_warnings=False
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   zero_enabled ................. True
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   zero_force_ds_cpu_optimizer .. True
+[2025-06-26 23:48:44,918] [INFO] [config.py:925:print]   zero_optimization_stage ...... 1
+[2025-06-26 23:48:44,918] [INFO] [config.py:911:print_user_config]   json = {
+    "optimizer": {
+        "type": "Adam", 
+        "params": {
+            "lr": 0.0005, 
+            "betas": [0.9, 0.999], 
+            "eps": 1e-08, 
+            "weight_decay": 0.0
+        }
+    }, 
+    "bf16": {
+        "enabled": true
+    }, 
+    "fp16": {
+        "enabled": false, 
+        "loss_scale": 0, 
+        "loss_scale_window": 1000, 
+        "initial_scale_power": 16, 
+        "hysteresis": 2, 
+        "min_loss_scale": 1
+    }, 
+    "zero_optimization": {
+        "stage": 1, 
+        "offload_optimizer": {
+            "device": "cpu", 
+            "pin_memory": true
+        }, 
+        "allgather_partitions": true, 
+        "allgather_bucket_size": 2.000000e+08, 
+        "overlap_comm": true, 
+        "reduce_scatter": true, 
+        "reduce_bucket_size": 2.000000e+08, 
+        "contiguous_gradients": true
+    }, 
+    "gradient_accumulation_steps": 1, 
+    "gradient_clipping": 1.0, 
+    "steps_per_print": inf, 
+    "train_batch_size": 100, 
+    "train_micro_batch_size_per_gpu": 25, 
+    "wall_clock_breakdown": false
+}
+[INFO|trainer.py:2145] 2025-06-26 23:48:44,920 >> ***** Running training *****
+[INFO|trainer.py:2146] 2025-06-26 23:48:44,920 >>   Num examples = 15,000
+[INFO|trainer.py:2147] 2025-06-26 23:48:44,920 >>   Num Epochs = 5
+[INFO|trainer.py:2148] 2025-06-26 23:48:44,920 >>   Instantaneous batch size per device = 25
+[INFO|trainer.py:2151] 2025-06-26 23:48:44,920 >>   Total train batch size (w. parallel, distributed & accumulation) = 100
+[INFO|trainer.py:2152] 2025-06-26 23:48:44,920 >>   Gradient Accumulation steps = 1
+[INFO|trainer.py:2153] 2025-06-26 23:48:44,920 >>   Total optimization steps = 750
+[INFO|trainer.py:2154] 2025-06-26 23:48:44,924 >>   Number of trainable parameters = 20,971,520
+[INFO|integration_utils.py:807] 2025-06-26 23:48:44,927 >> Automatic Weights & Biases logging enabled, to disable set os.environ["WANDB_DISABLED"] = "true"
+wandb: WARNING The `run_name` is currently set to the same value as `TrainingArguments.output_dir`. If this was not intended, please specify a different run name by setting the `TrainingArguments.run_name` parameter.
+wandb: Currently logged in as: sidharthpulipaka (indic-encoder) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
+wandb: Tracking run with wandb version 0.20.1
+wandb: Run data is saved locally in /home/iitm_admin/llmteam/mid-align/wandb/run-20250626_234845-s645lnzf
+wandb: Run `wandb offline` to turn off syncing.
+wandb: Syncing run ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/
+wandb: ⭐️ View project at https://wandb.ai/indic-encoder/midalign
+wandb: 🚀 View run at https://wandb.ai/indic-encoder/midalign/runs/s645lnzf
+
  0%|          | 0/750 [00:00<?, ?it/s][INFO|trainer.py:3831] 2025-06-26 23:48:46,992 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-26 23:48:46,992 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-26 23:48:46,992 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:01<00:07,  1.04it/s]
+
 30%|███       | 3/10 [00:04<00:10,  1.51s/it]
+
 40%|████      | 4/10 [00:06<00:09,  1.63s/it]
+
 50%|█████     | 5/10 [00:07<00:08,  1.70s/it]
+
 60%|██████    | 6/10 [00:09<00:06,  1.74s/it]
+
 70%|███████   | 7/10 [00:11<00:05,  1.77s/it]
+
 80%|████████  | 8/10 [00:13<00:03,  1.78s/it]
+
 90%|█████████ | 9/10 [00:15<00:01,  1.78s/it]
+
100%|██████████| 10/10 [00:18<00:00,  2.42s/it]
                                       
+
                                               
{'eval_loss': 0.3140127956867218, 'eval_accuracy': 0.21613092330239375, 'eval_runtime': 36.1586, 'eval_samples_per_second': 27.656, 'eval_steps_per_second': 0.277, 'epoch': 0}
+
  0%|          | 0/750 [00:36<?, ?it/s]
+
100%|██████████| 10/10 [00:24<00:00,  2.42s/it]
+
                                               
  0%|          | 1/750 [00:50<10:32:47, 50.69s/it]
                                                  
{'loss': 0.3212, 'grad_norm': 0.5906680822372437, 'learning_rate': 2.173913043478261e-05, 'epoch': 0.01}
+
  0%|          | 1/750 [00:50<10:32:47, 50.69s/it]
  0%|          | 2/750 [00:56<5:03:24, 24.34s/it] 
                                                 
{'loss': 0.3146, 'grad_norm': 0.5036028623580933, 'learning_rate': 4.347826086956522e-05, 'epoch': 0.01}
+
  0%|          | 2/750 [00:56<5:03:24, 24.34s/it]
  0%|          | 3/750 [01:02<3:17:49, 15.89s/it]
                                                 
{'loss': 0.326, 'grad_norm': 0.6123895049095154, 'learning_rate': 6.521739130434782e-05, 'epoch': 0.02}
+
  0%|          | 3/750 [01:02<3:17:49, 15.89s/it]
  1%|          | 4/750 [01:08<2:28:21, 11.93s/it]
                                                 
{'loss': 0.3315, 'grad_norm': 0.4841865003108978, 'learning_rate': 8.695652173913044e-05, 'epoch': 0.03}
+
  1%|          | 4/750 [01:08<2:28:21, 11.93s/it]
  1%|          | 5/750 [01:14<2:01:25,  9.78s/it]
                                                 
{'loss': 0.3074, 'grad_norm': 0.17575925588607788, 'learning_rate': 0.00010869565217391305, 'epoch': 0.03}
+
  1%|          | 5/750 [01:14<2:01:25,  9.78s/it]
  1%|          | 6/750 [01:20<1:44:51,  8.46s/it]
                                                 
{'loss': 0.2978, 'grad_norm': 0.15657196938991547, 'learning_rate': 0.00013043478260869564, 'epoch': 0.04}
+
  1%|          | 6/750 [01:20<1:44:51,  8.46s/it]
  1%|          | 7/750 [01:26<1:34:35,  7.64s/it]
                                                 
{'loss': 0.2925, 'grad_norm': 0.12743023037910461, 'learning_rate': 0.00015217391304347827, 'epoch': 0.05}
+
  1%|          | 7/750 [01:26<1:34:35,  7.64s/it]
  1%|          | 8/750 [01:32<1:27:42,  7.09s/it]
                                                 
{'loss': 0.3015, 'grad_norm': 0.9264193773269653, 'learning_rate': 0.00017391304347826088, 'epoch': 0.05}
+
  1%|          | 8/750 [01:32<1:27:42,  7.09s/it]
  1%|          | 9/750 [01:37<1:23:06,  6.73s/it]
                                                 
{'loss': 0.2809, 'grad_norm': 0.11622568219900131, 'learning_rate': 0.0001956521739130435, 'epoch': 0.06}
+
  1%|          | 9/750 [01:37<1:23:06,  6.73s/it]
  1%|▏         | 10/750 [01:43<1:19:53,  6.48s/it]
                                                  
{'loss': 0.2726, 'grad_norm': 0.10038203746080399, 'learning_rate': 0.0002173913043478261, 'epoch': 0.07}
+
  1%|▏         | 10/750 [01:43<1:19:53,  6.48s/it]
  1%|▏         | 11/750 [01:49<1:17:21,  6.28s/it]
                                                  
{'loss': 0.2686, 'grad_norm': 0.09309244155883789, 'learning_rate': 0.0002391304347826087, 'epoch': 0.07}
+
  1%|▏         | 11/750 [01:49<1:17:21,  6.28s/it]
  2%|▏         | 12/750 [01:55<1:16:19,  6.21s/it]
                                                  
{'loss': 0.2624, 'grad_norm': 0.08546415716409683, 'learning_rate': 0.0002608695652173913, 'epoch': 0.08}
+
  2%|▏         | 12/750 [01:55<1:16:19,  6.21s/it]
  2%|▏         | 13/750 [02:01<1:15:08,  6.12s/it]
                                                  
{'loss': 0.2869, 'grad_norm': 0.19671578705310822, 'learning_rate': 0.0002826086956521739, 'epoch': 0.09}
+
  2%|▏         | 13/750 [02:01<1:15:08,  6.12s/it]
  2%|▏         | 14/750 [02:07<1:14:29,  6.07s/it]
                                                  
{'loss': 0.2344, 'grad_norm': 0.08713918924331665, 'learning_rate': 0.00030434782608695655, 'epoch': 0.09}
+
  2%|▏         | 14/750 [02:07<1:14:29,  6.07s/it]
  2%|▏         | 15/750 [02:13<1:14:03,  6.05s/it]
                                                  
{'loss': 0.2675, 'grad_norm': 0.1005919948220253, 'learning_rate': 0.00032608695652173916, 'epoch': 0.1}
+
  2%|▏         | 15/750 [02:13<1:14:03,  6.05s/it]
  2%|▏         | 16/750 [02:19<1:14:05,  6.06s/it]
                                                  
{'loss': 0.2605, 'grad_norm': 0.0813712626695633, 'learning_rate': 0.00034782608695652176, 'epoch': 0.11}
+
  2%|▏         | 16/750 [02:19<1:14:05,  6.06s/it]
  2%|▏         | 17/750 [02:25<1:14:50,  6.13s/it]
                                                  
{'loss': 0.2862, 'grad_norm': 0.08247795701026917, 'learning_rate': 0.0003695652173913043, 'epoch': 0.11}
+
  2%|▏         | 17/750 [02:25<1:14:50,  6.13s/it]
  2%|▏         | 18/750 [02:31<1:14:09,  6.08s/it]
                                                  
{'loss': 0.2566, 'grad_norm': 0.085903599858284, 'learning_rate': 0.000391304347826087, 'epoch': 0.12}
+
  2%|▏         | 18/750 [02:31<1:14:09,  6.08s/it]
  3%|▎         | 19/750 [02:37<1:13:23,  6.02s/it]
                                                  
{'loss': 0.2463, 'grad_norm': 0.07887786626815796, 'learning_rate': 0.0004130434782608696, 'epoch': 0.13}
+
  3%|▎         | 19/750 [02:37<1:13:23,  6.02s/it]
  3%|▎         | 20/750 [02:43<1:13:09,  6.01s/it]
                                                  
{'loss': 0.2608, 'grad_norm': 0.07987634092569351, 'learning_rate': 0.0004347826086956522, 'epoch': 0.13}
+
  3%|▎         | 20/750 [02:43<1:13:09,  6.01s/it]
  3%|▎         | 21/750 [02:49<1:12:41,  5.98s/it]
                                                  
{'loss': 0.2465, 'grad_norm': 0.08308937400579453, 'learning_rate': 0.00045652173913043474, 'epoch': 0.14}
+
  3%|▎         | 21/750 [02:49<1:12:41,  5.98s/it]
  3%|▎         | 22/750 [02:55<1:12:09,  5.95s/it]
                                                  
{'loss': 0.25, 'grad_norm': 0.08658931404352188, 'learning_rate': 0.0004782608695652174, 'epoch': 0.15}
+
  3%|▎         | 22/750 [02:55<1:12:09,  5.95s/it]
  3%|▎         | 23/750 [03:01<1:12:03,  5.95s/it]
                                                  
{'loss': 0.2414, 'grad_norm': 0.07640273869037628, 'learning_rate': 0.0005, 'epoch': 0.15}
+
  3%|▎         | 23/750 [03:01<1:12:03,  5.95s/it]
  3%|▎         | 24/750 [03:07<1:12:11,  5.97s/it]
                                                  
{'loss': 0.2445, 'grad_norm': 0.0850343406200409, 'learning_rate': 0.0004894725051862804, 'epoch': 0.16}
+
  3%|▎         | 24/750 [03:07<1:12:11,  5.97s/it]
  3%|▎         | 25/750 [03:13<1:12:07,  5.97s/it]
                                                  
{'loss': 0.2519, 'grad_norm': 0.08253505825996399, 'learning_rate': 0.00047958315233127195, 'epoch': 0.17}
+
  3%|▎         | 25/750 [03:13<1:12:07,  5.97s/it]
  3%|▎         | 26/750 [03:19<1:11:57,  5.96s/it]
                                                  
{'loss': 0.2638, 'grad_norm': 0.08638802170753479, 'learning_rate': 0.0004702699715629801, 'epoch': 0.17}
+
  3%|▎         | 26/750 [03:19<1:11:57,  5.96s/it]
  4%|▎         | 27/750 [03:25<1:11:32,  5.94s/it]
                                                  
{'loss': 0.2404, 'grad_norm': 0.08205699175596237, 'learning_rate': 0.00046147910349544864, 'epoch': 0.18}
+
  4%|▎         | 27/750 [03:25<1:11:32,  5.94s/it]
  4%|▎         | 28/750 [03:30<1:10:20,  5.85s/it]
                                                  
{'loss': 0.2694, 'grad_norm': 0.08746669441461563, 'learning_rate': 0.00045316348358748285, 'epoch': 0.19}
+
  4%|▎         | 28/750 [03:30<1:10:20,  5.85s/it]
  4%|▍         | 29/750 [03:36<1:10:40,  5.88s/it]
                                                  
{'loss': 0.2473, 'grad_norm': 0.08038605004549026, 'learning_rate': 0.00044528177828086066, 'epoch': 0.19}
+
  4%|▍         | 29/750 [03:36<1:10:40,  5.88s/it]
  4%|▍         | 30/750 [03:42<1:10:46,  5.90s/it]
                                                  
{'loss': 0.2187, 'grad_norm': 0.07426661252975464, 'learning_rate': 0.0004377975178854565, 'epoch': 0.2}
+
  4%|▍         | 30/750 [03:42<1:10:46,  5.90s/it]
  4%|▍         | 31/750 [03:48<1:10:50,  5.91s/it]
                                                  
{'loss': 0.2407, 'grad_norm': 0.0821617841720581, 'learning_rate': 0.0004306783846070544, 'epoch': 0.21}
+
  4%|▍         | 31/750 [03:48<1:10:50,  5.91s/it]
  4%|▍         | 32/750 [03:54<1:11:01,  5.94s/it]
                                                  
{'loss': 0.2354, 'grad_norm': 0.07908231019973755, 'learning_rate': 0.00042389562394532925, 'epoch': 0.21}
+
  4%|▍         | 32/750 [03:54<1:11:01,  5.94s/it]
  4%|▍         | 33/750 [04:00<1:10:49,  5.93s/it]
                                                  
{'loss': 0.25, 'grad_norm': 0.0851181373000145, 'learning_rate': 0.00041742355496836097, 'epoch': 0.22}
+
  4%|▍         | 33/750 [04:00<1:10:49,  5.93s/it]
  5%|▍         | 34/750 [04:06<1:10:43,  5.93s/it]
                                                  
{'loss': 0.2568, 'grad_norm': 0.08554762601852417, 'learning_rate': 0.00041123916041498716, 'epoch': 0.23}
+
  5%|▍         | 34/750 [04:06<1:10:43,  5.93s/it]
  5%|▍         | 35/750 [04:12<1:10:38,  5.93s/it]
                                                  
{'loss': 0.2252, 'grad_norm': 0.07662541419267654, 'learning_rate': 0.00040532174168888874, 'epoch': 0.23}
+
  5%|▍         | 35/750 [04:12<1:10:38,  5.93s/it]
  5%|▍         | 36/750 [04:18<1:12:08,  6.06s/it]
                                                  
{'loss': 0.2524, 'grad_norm': 0.08561822772026062, 'learning_rate': 0.00039965262694272664, 'epoch': 0.24}
+
  5%|▍         | 36/750 [04:18<1:12:08,  6.06s/it]
  5%|▍         | 37/750 [04:24<1:09:06,  5.82s/it]
                                                  
{'loss': 0.2463, 'grad_norm': 0.09797465801239014, 'learning_rate': 0.00039421492285986, 'epoch': 0.25}
+
  5%|▍         | 37/750 [04:24<1:09:06,  5.82s/it]
  5%|▌         | 38/750 [04:30<1:09:34,  5.86s/it]
                                                  
{'loss': 0.2568, 'grad_norm': 0.07547479122877121, 'learning_rate': 0.00038899330260774953, 'epoch': 0.25}
+
  5%|▌         | 38/750 [04:30<1:09:34,  5.86s/it]
  5%|▌         | 39/750 [04:35<1:09:14,  5.84s/it]
                                                  
{'loss': 0.2354, 'grad_norm': 0.07932130247354507, 'learning_rate': 0.00038397382389415225, 'epoch': 0.26}
+
  5%|▌         | 39/750 [04:35<1:09:14,  5.84s/it]
  5%|▌         | 40/750 [04:41<1:09:33,  5.88s/it]
                                                  
{'loss': 0.238, 'grad_norm': 0.07694815844297409, 'learning_rate': 0.00037914377220257757, 'epoch': 0.27}
+
  5%|▌         | 40/750 [04:41<1:09:33,  5.88s/it]
  5%|▌         | 41/750 [04:47<1:09:41,  5.90s/it]
                                                  
{'loss': 0.2461, 'grad_norm': 0.07635044306516647, 'learning_rate': 0.0003744915251898558, 'epoch': 0.27}
+
  5%|▌         | 41/750 [04:47<1:09:41,  5.90s/it]
  6%|▌         | 42/750 [04:53<1:09:49,  5.92s/it]
                                                  
{'loss': 0.2418, 'grad_norm': 0.07495218515396118, 'learning_rate': 0.00037000643495047747, 'epoch': 0.28}
+
  6%|▌         | 42/750 [04:53<1:09:49,  5.92s/it]
  6%|▌         | 43/750 [04:59<1:10:03,  5.95s/it]
                                                  
{'loss': 0.2588, 'grad_norm': 0.08053979277610779, 'learning_rate': 0.00036567872543061367, 'epoch': 0.29}
+
  6%|▌         | 43/750 [04:59<1:10:03,  5.95s/it]
  6%|▌         | 44/750 [05:05<1:09:57,  5.95s/it]
                                                  
{'loss': 0.2398, 'grad_norm': 0.07696537673473358, 'learning_rate': 0.0003614994027406106, 'epoch': 0.29}
+
  6%|▌         | 44/750 [05:05<1:09:57,  5.95s/it]
  6%|▌         | 45/750 [05:11<1:09:46,  5.94s/it]
                                                  
{'loss': 0.2206, 'grad_norm': 0.07416495680809021, 'learning_rate': 0.0003574601764921203, 'epoch': 0.3}
+
  6%|▌         | 45/750 [05:11<1:09:46,  5.94s/it]
  6%|▌         | 46/750 [05:17<1:09:50,  5.95s/it]
                                                  
{'loss': 0.2425, 'grad_norm': 0.07514647394418716, 'learning_rate': 0.00035355339059327376, 'epoch': 0.31}
+
  6%|▌         | 46/750 [05:17<1:09:50,  5.95s/it]
  6%|▋         | 47/750 [05:23<1:09:44,  5.95s/it]
                                                  
{'loss': 0.243, 'grad_norm': 0.07844290137290955, 'learning_rate': 0.0003497719621866722, 'epoch': 0.31}
+
  6%|▋         | 47/750 [05:23<1:09:44,  5.95s/it]
  6%|▋         | 48/750 [05:29<1:09:15,  5.92s/it]
                                                  
{'loss': 0.2506, 'grad_norm': 0.08102063834667206, 'learning_rate': 0.0003461093276215865, 'epoch': 0.32}
+
  6%|▋         | 48/750 [05:29<1:09:15,  5.92s/it]
  7%|▋         | 49/750 [05:35<1:08:59,  5.90s/it]
                                                  
{'loss': 0.2459, 'grad_norm': 0.07465319335460663, 'learning_rate': 0.0003425593945223371, 'epoch': 0.33}
+
  7%|▋         | 49/750 [05:35<1:08:59,  5.90s/it]
  7%|▋         | 50/750 [05:41<1:09:07,  5.92s/it]
                                                  
{'loss': 0.2296, 'grad_norm': 0.07176068425178528, 'learning_rate': 0.0003391164991562634, 'epoch': 0.33}
+
  7%|▋         | 50/750 [05:41<1:09:07,  5.92s/it]
  7%|▋         | 51/750 [05:47<1:08:48,  5.91s/it]
                                                  
{'loss': 0.2137, 'grad_norm': 0.07323901355266571, 'learning_rate': 0.00033577536842242567, 'epoch': 0.34}
+
  7%|▋         | 51/750 [05:47<1:08:48,  5.91s/it]
  7%|▋         | 52/750 [05:53<1:08:44,  5.91s/it]
                                                  
{'loss': 0.2297, 'grad_norm': 0.07271301746368408, 'learning_rate': 0.0003325310858805882, 'epoch': 0.35}
+
  7%|▋         | 52/750 [05:53<1:08:44,  5.91s/it]
  7%|▋         | 53/750 [05:59<1:08:38,  5.91s/it]
                                                  
{'loss': 0.2419, 'grad_norm': 0.07504609227180481, 'learning_rate': 0.0003293790613225677, 'epoch': 0.35}
+
  7%|▋         | 53/750 [05:59<1:08:38,  5.91s/it]
  7%|▋         | 54/750 [06:05<1:09:08,  5.96s/it]
                                                  
{'loss': 0.2379, 'grad_norm': 0.07151075452566147, 'learning_rate': 0.0003263150034575203, 'epoch': 0.36}
+
  7%|▋         | 54/750 [06:05<1:09:08,  5.96s/it]
  7%|▋         | 55/750 [06:11<1:08:52,  5.95s/it]
                                                  
{'loss': 0.2266, 'grad_norm': 0.07085525244474411, 'learning_rate': 0.00032333489534143163, 'epoch': 0.37}
+
  7%|▋         | 55/750 [06:11<1:08:52,  5.95s/it]
  7%|▋         | 56/750 [06:16<1:08:51,  5.95s/it]
                                                  
{'loss': 0.2255, 'grad_norm': 0.07336307317018509, 'learning_rate': 0.0003204349722308279, 'epoch': 0.37}
+
  7%|▋         | 56/750 [06:16<1:08:51,  5.95s/it]
  8%|▊         | 57/750 [06:22<1:08:38,  5.94s/it]
                                                  
{'loss': 0.2336, 'grad_norm': 0.07599075138568878, 'learning_rate': 0.0003176117015830118, 'epoch': 0.38}
+
  8%|▊         | 57/750 [06:22<1:08:38,  5.94s/it]
  8%|▊         | 58/750 [06:28<1:08:27,  5.94s/it]
                                                  
{'loss': 0.2239, 'grad_norm': 0.07336984574794769, 'learning_rate': 0.00031486176496120126, 'epoch': 0.39}
+
  8%|▊         | 58/750 [06:28<1:08:27,  5.94s/it]
  8%|▊         | 59/750 [06:34<1:08:27,  5.94s/it]
                                                  
{'loss': 0.2139, 'grad_norm': 0.07115612924098969, 'learning_rate': 0.0003121820416337943, 'epoch': 0.39}
+
  8%|▊         | 59/750 [06:34<1:08:27,  5.94s/it]
  8%|▊         | 60/750 [06:40<1:08:43,  5.98s/it]
                                                  
{'loss': 0.2362, 'grad_norm': 0.07719278335571289, 'learning_rate': 0.00030956959368344517, 'epoch': 0.4}
+
  8%|▊         | 60/750 [06:40<1:08:43,  5.98s/it]
  8%|▊         | 61/750 [06:46<1:08:27,  5.96s/it]
                                                  
{'loss': 0.2469, 'grad_norm': 0.07207552343606949, 'learning_rate': 0.00030702165246439413, 'epoch': 0.41}
+
  8%|▊         | 61/750 [06:46<1:08:27,  5.96s/it]
  8%|▊         | 62/750 [06:52<1:08:15,  5.95s/it]
                                                  
{'loss': 0.2161, 'grad_norm': 0.07065242528915405, 'learning_rate': 0.0003045356062661163, 'epoch': 0.41}
+
  8%|▊         | 62/750 [06:52<1:08:15,  5.95s/it]
  8%|▊         | 63/750 [06:58<1:08:09,  5.95s/it]
                                                  
{'loss': 0.252, 'grad_norm': 0.07868544012308121, 'learning_rate': 0.0003021089890583219, 'epoch': 0.42}
+
  8%|▊         | 63/750 [06:58<1:08:09,  5.95s/it]
  9%|▊         | 64/750 [07:04<1:08:00,  5.95s/it]
                                                  
{'loss': 0.2523, 'grad_norm': 0.07709171622991562, 'learning_rate': 0.00029973947020704494, 'epoch': 0.43}
+
  9%|▊         | 64/750 [07:04<1:08:00,  5.95s/it]
  9%|▊         | 65/750 [07:10<1:07:34,  5.92s/it]
                                                  
{'loss': 0.2216, 'grad_norm': 0.07212568819522858, 'learning_rate': 0.0002974248450643262, 'epoch': 0.43}
+
  9%|▊         | 65/750 [07:10<1:07:34,  5.92s/it]
  9%|▉         | 66/750 [07:16<1:07:36,  5.93s/it]
                                                  
{'loss': 0.2284, 'grad_norm': 0.0752672329545021, 'learning_rate': 0.0002951630263451236, 'epoch': 0.44}
+
  9%|▉         | 66/750 [07:16<1:07:36,  5.93s/it]
  9%|▉         | 67/750 [07:22<1:07:38,  5.94s/it]
                                                  
{'loss': 0.2303, 'grad_norm': 0.07730327546596527, 'learning_rate': 0.0002929520362147839, 'epoch': 0.45}
+
  9%|▉         | 67/750 [07:22<1:07:38,  5.94s/it]
  9%|▉         | 68/750 [07:28<1:07:19,  5.92s/it]
                                                  
{'loss': 0.1982, 'grad_norm': 0.07627134770154953, 'learning_rate': 0.00029078999901889984, 'epoch': 0.45}
+
  9%|▉         | 68/750 [07:28<1:07:19,  5.92s/it]
  9%|▉         | 69/750 [07:34<1:07:19,  5.93s/it]
                                                  
{'loss': 0.2296, 'grad_norm': 0.07409168034791946, 'learning_rate': 0.00028867513459481295, 'epoch': 0.46}
+
  9%|▉         | 69/750 [07:34<1:07:19,  5.93s/it]
  9%|▉         | 70/750 [07:40<1:07:09,  5.93s/it]
                                                  
{'loss': 0.2038, 'grad_norm': 0.07780491560697556, 'learning_rate': 0.0002866057521105554, 'epoch': 0.47}
+
  9%|▉         | 70/750 [07:40<1:07:09,  5.93s/it]
  9%|▉         | 71/750 [07:46<1:07:10,  5.94s/it]
                                                  
{'loss': 0.2472, 'grad_norm': 0.08318985998630524, 'learning_rate': 0.0002845802443827711, 'epoch': 0.47}
+
  9%|▉         | 71/750 [07:46<1:07:10,  5.94s/it]
 10%|▉         | 72/750 [07:52<1:07:25,  5.97s/it]
                                                  
{'loss': 0.2101, 'grad_norm': 0.0681476816534996, 'learning_rate': 0.0002825970826302195, 'epoch': 0.48}
+
 10%|▉         | 72/750 [07:52<1:07:25,  5.97s/it]
 10%|▉         | 73/750 [07:58<1:07:07,  5.95s/it]
                                                  
{'loss': 0.2138, 'grad_norm': 0.07284767180681229, 'learning_rate': 0.0002806548116239436, 'epoch': 0.49}
+
 10%|▉         | 73/750 [07:58<1:07:07,  5.95s/it]
 10%|▉         | 74/750 [08:03<1:06:56,  5.94s/it]
                                                  
{'loss': 0.2358, 'grad_norm': 0.07210768014192581, 'learning_rate': 0.00027875204519913876, 'epoch': 0.49}
+
 10%|▉         | 74/750 [08:03<1:06:56,  5.94s/it]
 10%|█         | 75/750 [08:09<1:06:43,  5.93s/it]
                                                  
{'loss': 0.2351, 'grad_norm': 0.07658939808607101, 'learning_rate': 0.00027688746209726916, 'epoch': 0.5}
+
 10%|█         | 75/750 [08:09<1:06:43,  5.93s/it]
 10%|█         | 76/750 [08:15<1:06:47,  5.95s/it]
                                                  
{'loss': 0.2427, 'grad_norm': 0.07850906997919083, 'learning_rate': 0.0002750598021100904, 'epoch': 0.51}
+
 10%|█         | 76/750 [08:15<1:06:47,  5.95s/it]
 10%|█         | 77/750 [08:21<1:06:49,  5.96s/it]
                                                  
{'loss': 0.2396, 'grad_norm': 0.07813714444637299, 'learning_rate': 0.0002732678625000106, 'epoch': 0.51}
+
 10%|█         | 77/750 [08:21<1:06:49,  5.96s/it]
 10%|█         | 78/750 [08:27<1:06:39,  5.95s/it]
                                                  
{'loss': 0.238, 'grad_norm': 0.07168495655059814, 'learning_rate': 0.00027151049467368427, 'epoch': 0.52}
+
 10%|█         | 78/750 [08:27<1:06:39,  5.95s/it]
 11%|█         | 79/750 [08:33<1:05:29,  5.86s/it]
                                                  
{'loss': 0.2303, 'grad_norm': 0.08337289094924927, 'learning_rate': 0.00026978660108793815, 'epoch': 0.53}
+
 11%|█         | 79/750 [08:33<1:05:29,  5.86s/it]
 11%|█         | 80/750 [08:39<1:05:30,  5.87s/it]
                                                  
{'loss': 0.2269, 'grad_norm': 0.07532817870378494, 'learning_rate': 0.00026809513236909023, 'epoch': 0.53}
+
 11%|█         | 80/750 [08:39<1:05:30,  5.87s/it]
 11%|█         | 81/750 [08:45<1:05:53,  5.91s/it]
                                                  
{'loss': 0.226, 'grad_norm': 0.06945842504501343, 'learning_rate': 0.00026643508462848443, 'epoch': 0.54}
+
 11%|█         | 81/750 [08:45<1:05:53,  5.91s/it]
 11%|█         | 82/750 [08:51<1:05:44,  5.91s/it]
                                                  
{'loss': 0.2117, 'grad_norm': 0.0758850947022438, 'learning_rate': 0.00026480549695863983, 'epoch': 0.55}
+
 11%|█         | 82/750 [08:51<1:05:44,  5.91s/it]
 11%|█         | 83/750 [08:57<1:05:52,  5.93s/it]
                                                  
{'loss': 0.2656, 'grad_norm': 0.07941056042909622, 'learning_rate': 0.000263205449095825, 'epoch': 0.55}
+
 11%|█         | 83/750 [08:57<1:05:52,  5.93s/it]
 11%|█         | 84/750 [09:03<1:05:55,  5.94s/it]
                                                  
{'loss': 0.2323, 'grad_norm': 0.07193499803543091, 'learning_rate': 0.0002616340592361418, 'epoch': 0.56}
+
 11%|█         | 84/750 [09:03<1:05:55,  5.94s/it]
 11%|█▏        | 85/750 [09:09<1:05:39,  5.92s/it]
                                                  
{'loss': 0.2139, 'grad_norm': 0.0777997300028801, 'learning_rate': 0.00026009048199334285, 'epoch': 0.57}
+
 11%|█▏        | 85/750 [09:09<1:05:39,  5.92s/it]
 11%|█▏        | 86/750 [09:15<1:05:46,  5.94s/it]
                                                  
{'loss': 0.2277, 'grad_norm': 0.07722801715135574, 'learning_rate': 0.0002585739064876405, 'epoch': 0.57}
+
 11%|█▏        | 86/750 [09:15<1:05:46,  5.94s/it]
 12%|█▏        | 87/750 [09:21<1:05:54,  5.96s/it]
                                                  
{'loss': 0.2055, 'grad_norm': 0.07862409204244614, 'learning_rate': 0.0002570835545556902, 'epoch': 0.58}
+
 12%|█▏        | 87/750 [09:21<1:05:54,  5.96s/it]
 12%|█▏        | 88/750 [09:26<1:05:45,  5.96s/it]
                                                  
{'loss': 0.2331, 'grad_norm': 0.07638678699731827, 'learning_rate': 0.00025561867907277256, 'epoch': 0.59}
+
 12%|█▏        | 88/750 [09:26<1:05:45,  5.96s/it]
 12%|█▏        | 89/750 [09:32<1:05:26,  5.94s/it]
                                                  
{'loss': 0.2155, 'grad_norm': 0.07443062216043472, 'learning_rate': 0.0002541785623789578, 'epoch': 0.59}
+
 12%|█▏        | 89/750 [09:32<1:05:26,  5.94s/it]
 12%|█▏        | 90/750 [09:38<1:05:08,  5.92s/it]
                                                  
{'loss': 0.2069, 'grad_norm': 0.0773666501045227, 'learning_rate': 0.00025276251480171836, 'epoch': 0.6}
+
 12%|█▏        | 90/750 [09:38<1:05:08,  5.92s/it]
 12%|█▏        | 91/750 [09:44<1:05:38,  5.98s/it]
                                                  
{'loss': 0.2331, 'grad_norm': 0.07648374140262604, 'learning_rate': 0.00025136987326808516, 'epoch': 0.61}
+
 12%|█▏        | 91/750 [09:44<1:05:38,  5.98s/it]
 12%|█▏        | 92/750 [09:51<1:06:16,  6.04s/it]
                                                  
{'loss': 0.2335, 'grad_norm': 0.0779750794172287, 'learning_rate': 0.00025, 'epoch': 0.61}
+
 12%|█▏        | 92/750 [09:51<1:06:16,  6.04s/it]
 12%|█▏        | 93/750 [09:56<1:05:45,  6.01s/it]
                                                  
{'loss': 0.2539, 'grad_norm': 0.07936685532331467, 'learning_rate': 0.00024865228128703607, 'epoch': 0.62}
+
 12%|█▏        | 93/750 [09:56<1:05:45,  6.01s/it]
 13%|█▎        | 94/750 [10:02<1:05:13,  5.97s/it]
                                                  
{'loss': 0.2101, 'grad_norm': 0.07413629442453384, 'learning_rate': 0.0002473261263311206, 'epoch': 0.63}
+
 13%|█▎        | 94/750 [10:02<1:05:13,  5.97s/it]
 13%|█▎        | 95/750 [10:08<1:04:54,  5.95s/it]
                                                  
{'loss': 0.2213, 'grad_norm': 0.0760723352432251, 'learning_rate': 0.00024602096615832094, 'epoch': 0.63}
+
 13%|█▎        | 95/750 [10:08<1:04:54,  5.95s/it]
 13%|█▎        | 96/750 [10:14<1:04:55,  5.96s/it]
                                                  
{'loss': 0.243, 'grad_norm': 0.07862426340579987, 'learning_rate': 0.0002447362525931402, 'epoch': 0.64}
+
 13%|█▎        | 96/750 [10:14<1:04:55,  5.96s/it]
 13%|█▎        | 97/750 [10:20<1:04:30,  5.93s/it]
                                                  
{'loss': 0.1999, 'grad_norm': 0.08154033869504929, 'learning_rate': 0.0002434714572911246, 'epoch': 0.65}
+
 13%|█▎        | 97/750 [10:20<1:04:30,  5.93s/it]
 13%|█▎        | 98/750 [10:26<1:04:31,  5.94s/it]
                                                  
{'loss': 0.2419, 'grad_norm': 0.07939109206199646, 'learning_rate': 0.00024222607082590244, 'epoch': 0.65}
+
 13%|█▎        | 98/750 [10:26<1:04:31,  5.94s/it]
 13%|█▎        | 99/750 [10:32<1:04:34,  5.95s/it]
                                                  
{'loss': 0.2275, 'grad_norm': 0.08373291790485382, 'learning_rate': 0.00024099960182707373, 'epoch': 0.66}
+
 13%|█▎        | 99/750 [10:32<1:04:34,  5.95s/it]
 13%|█▎        | 100/750 [10:38<1:04:18,  5.94s/it]
                                                   
{'loss': 0.2212, 'grad_norm': 0.07675233483314514, 'learning_rate': 0.00023979157616563598, 'epoch': 0.67}
+
 13%|█▎        | 100/750 [10:38<1:04:18,  5.94s/it]
 13%|█▎        | 101/750 [10:44<1:04:18,  5.95s/it]
                                                   
{'loss': 0.2361, 'grad_norm': 0.07895852625370026, 'learning_rate': 0.00023860153618387902, 'epoch': 0.67}
+
 13%|█▎        | 101/750 [10:44<1:04:18,  5.95s/it]
 14%|█▎        | 102/750 [10:50<1:03:56,  5.92s/it]
                                                   
{'loss': 0.2065, 'grad_norm': 0.0748487189412117, 'learning_rate': 0.00023742903996690855, 'epoch': 0.68}
+
 14%|█▎        | 102/750 [10:50<1:03:56,  5.92s/it]
 14%|█▎        | 103/750 [10:56<1:03:35,  5.90s/it]
                                                   
{'loss': 0.2222, 'grad_norm': 0.07482165098190308, 'learning_rate': 0.00023627366065316423, 'epoch': 0.69}
+
 14%|█▎        | 103/750 [10:56<1:03:35,  5.90s/it]
 14%|█▍        | 104/750 [11:02<1:03:37,  5.91s/it]
                                                   
{'loss': 0.2196, 'grad_norm': 0.07298839837312698, 'learning_rate': 0.00023513498578149006, 'epoch': 0.69}
+
 14%|█▍        | 104/750 [11:02<1:03:37,  5.91s/it]
 14%|█▍        | 105/750 [11:07<1:03:43,  5.93s/it]
                                                   
{'loss': 0.2278, 'grad_norm': 0.07794521003961563, 'learning_rate': 0.0002340126166724879, 'epoch': 0.7}
+
 14%|█▍        | 105/750 [11:07<1:03:43,  5.93s/it]
 14%|█▍        | 106/750 [11:13<1:03:25,  5.91s/it]
                                                   
{'loss': 0.2387, 'grad_norm': 0.07902726531028748, 'learning_rate': 0.00023290616784204732, 'epoch': 0.71}
+
 14%|█▍        | 106/750 [11:13<1:03:25,  5.91s/it]
 14%|█▍        | 107/750 [11:19<1:03:30,  5.93s/it]
                                                   
{'loss': 0.2317, 'grad_norm': 0.08068089932203293, 'learning_rate': 0.00023181526644509276, 'epoch': 0.71}
+
 14%|█▍        | 107/750 [11:19<1:03:30,  5.93s/it]
 14%|█▍        | 108/750 [11:25<1:03:31,  5.94s/it]
                                                   
{'loss': 0.2385, 'grad_norm': 0.08232149481773376, 'learning_rate': 0.00023073955174772432, 'epoch': 0.72}
+
 14%|█▍        | 108/750 [11:25<1:03:31,  5.94s/it]
 15%|█▍        | 109/750 [11:31<1:03:24,  5.93s/it]
                                                   
{'loss': 0.2115, 'grad_norm': 0.07577244937419891, 'learning_rate': 0.0002296786746260559, 'epoch': 0.73}
+
 15%|█▍        | 109/750 [11:31<1:03:24,  5.93s/it]
 15%|█▍        | 110/750 [11:37<1:03:32,  5.96s/it]
                                                   
{'loss': 0.2365, 'grad_norm': 0.08440236747264862, 'learning_rate': 0.00022863229709016894, 'epoch': 0.73}
+
 15%|█▍        | 110/750 [11:37<1:03:32,  5.96s/it]
 15%|█▍        | 111/750 [11:43<1:02:21,  5.86s/it]
                                                   
{'loss': 0.2111, 'grad_norm': 0.0803331509232521, 'learning_rate': 0.00022760009183170774, 'epoch': 0.74}
+
 15%|█▍        | 111/750 [11:43<1:02:21,  5.86s/it]
 15%|█▍        | 112/750 [11:49<1:02:21,  5.86s/it]
                                                   
{'loss': 0.2414, 'grad_norm': 0.083428293466568, 'learning_rate': 0.00022658174179374143, 'epoch': 0.75}
+
 15%|█▍        | 112/750 [11:49<1:02:21,  5.86s/it]
 15%|█▌        | 113/750 [11:55<1:02:16,  5.87s/it]
                                                   
{'loss': 0.2002, 'grad_norm': 0.07875760644674301, 'learning_rate': 0.0002255769397616086, 'epoch': 0.75}
+
 15%|█▌        | 113/750 [11:55<1:02:16,  5.87s/it]
 15%|█▌        | 114/750 [12:01<1:02:35,  5.90s/it]
                                                   
{'loss': 0.2112, 'grad_norm': 0.07795742899179459, 'learning_rate': 0.00022458538797354575, 'epoch': 0.76}
+
 15%|█▌        | 114/750 [12:01<1:02:35,  5.90s/it]
 15%|█▌        | 115/750 [12:06<1:02:23,  5.90s/it]
                                                   
{'loss': 0.2177, 'grad_norm': 0.08177853375673294, 'learning_rate': 0.00022360679774997898, 'epoch': 0.77}
+
 15%|█▌        | 115/750 [12:06<1:02:23,  5.90s/it]
 15%|█▌        | 116/750 [12:12<1:02:29,  5.91s/it]
                                                   
{'loss': 0.2135, 'grad_norm': 0.08486494421958923, 'learning_rate': 0.00022264088914043033, 'epoch': 0.77}
+
 15%|█▌        | 116/750 [12:12<1:02:29,  5.91s/it]
 16%|█▌        | 117/750 [12:18<1:02:27,  5.92s/it]
                                                   
{'loss': 0.2142, 'grad_norm': 0.07922283560037613, 'learning_rate': 0.00022168739058705874, 'epoch': 0.78}
+
 16%|█▌        | 117/750 [12:18<1:02:27,  5.92s/it]
 16%|█▌        | 118/750 [12:24<1:02:36,  5.94s/it]
                                                   
{'loss': 0.2207, 'grad_norm': 0.07663120329380035, 'learning_rate': 0.00022074603860391704, 'epoch': 0.79}
+
 16%|█▌        | 118/750 [12:24<1:02:36,  5.94s/it]
 16%|█▌        | 119/750 [12:30<1:02:14,  5.92s/it]
                                                   
{'loss': 0.2342, 'grad_norm': 0.07658910006284714, 'learning_rate': 0.00021981657747106436, 'epoch': 0.79}
+
 16%|█▌        | 119/750 [12:30<1:02:14,  5.92s/it]
 16%|█▌        | 120/750 [12:36<1:01:54,  5.90s/it]
                                                   
{'loss': 0.2152, 'grad_norm': 0.07727257162332535, 'learning_rate': 0.00021889875894272825, 'epoch': 0.8}
+
 16%|█▌        | 120/750 [12:36<1:01:54,  5.90s/it]
 16%|█▌        | 121/750 [12:42<1:01:38,  5.88s/it]
                                                   
{'loss': 0.2415, 'grad_norm': 0.08518479764461517, 'learning_rate': 0.00021799234196875997, 'epoch': 0.81}
+
 16%|█▌        | 121/750 [12:42<1:01:38,  5.88s/it]
 16%|█▋        | 122/750 [12:48<1:01:52,  5.91s/it]
                                                   
{'loss': 0.214, 'grad_norm': 0.08320233225822449, 'learning_rate': 0.0002170970924286726, 'epoch': 0.81}
+
 16%|█▋        | 122/750 [12:48<1:01:52,  5.91s/it]
 16%|█▋        | 123/750 [12:54<1:01:54,  5.92s/it]
                                                   
{'loss': 0.2095, 'grad_norm': 0.07455971837043762, 'learning_rate': 0.00021621278287759676, 'epoch': 0.82}
+
 16%|█▋        | 123/750 [12:54<1:01:54,  5.92s/it]
 17%|█▋        | 124/750 [13:00<1:01:57,  5.94s/it]
                                                   
{'loss': 0.2204, 'grad_norm': 0.07474488019943237, 'learning_rate': 0.0002153391923035272, 'epoch': 0.83}
+
 17%|█▋        | 124/750 [13:00<1:01:57,  5.94s/it]
 17%|█▋        | 125/750 [13:06<1:01:30,  5.90s/it]
                                                   
{'loss': 0.2285, 'grad_norm': 0.08217088133096695, 'learning_rate': 0.00021447610589527216, 'epoch': 0.83}
+
 17%|█▋        | 125/750 [13:06<1:01:30,  5.90s/it]
 17%|█▋        | 126/750 [13:12<1:01:29,  5.91s/it]
                                                   
{'loss': 0.2242, 'grad_norm': 0.08691132813692093, 'learning_rate': 0.0002136233148205519, 'epoch': 0.84}
+
 17%|█▋        | 126/750 [13:12<1:01:29,  5.91s/it]
 17%|█▋        | 127/750 [13:17<1:01:23,  5.91s/it]
                                                   
{'loss': 0.2304, 'grad_norm': 0.08113974332809448, 'learning_rate': 0.00021278061601372693, 'epoch': 0.85}
+
 17%|█▋        | 127/750 [13:17<1:01:23,  5.91s/it]
 17%|█▋        | 128/750 [13:23<1:00:58,  5.88s/it]
                                                   
{'loss': 0.2495, 'grad_norm': 0.09231173992156982, 'learning_rate': 0.00021194781197266462, 'epoch': 0.85}
+
 17%|█▋        | 128/750 [13:23<1:00:58,  5.88s/it]
 17%|█▋        | 129/750 [13:29<1:01:44,  5.96s/it]
                                                   
{'loss': 0.2305, 'grad_norm': 0.08543296903371811, 'learning_rate': 0.00021112471056428407, 'epoch': 0.86}
+
 17%|█▋        | 129/750 [13:29<1:01:44,  5.96s/it]
 17%|█▋        | 130/750 [13:35<1:01:19,  5.93s/it]
                                                   
{'loss': 0.2262, 'grad_norm': 0.07670380175113678, 'learning_rate': 0.00021031112483834333, 'epoch': 0.87}
+
 17%|█▋        | 130/750 [13:35<1:01:19,  5.93s/it]
 17%|█▋        | 131/750 [13:41<1:01:16,  5.94s/it]
                                                   
{'loss': 0.2127, 'grad_norm': 0.08053433150053024, 'learning_rate': 0.00020950687284906043, 'epoch': 0.87}
+
 17%|█▋        | 131/750 [13:41<1:01:16,  5.94s/it]
 18%|█▊        | 132/750 [13:47<1:00:47,  5.90s/it]
                                                   
{'loss': 0.214, 'grad_norm': 0.08274922519922256, 'learning_rate': 0.00020871177748418049, 'epoch': 0.88}
+
 18%|█▊        | 132/750 [13:47<1:00:47,  5.90s/it]
 18%|█▊        | 133/750 [13:53<1:00:45,  5.91s/it]
                                                   
{'loss': 0.1953, 'grad_norm': 0.07750532031059265, 'learning_rate': 0.00020792566630112533, 'epoch': 0.89}
+
 18%|█▊        | 133/750 [13:53<1:00:45,  5.91s/it]
 18%|█▊        | 134/750 [13:59<1:00:30,  5.89s/it]
                                                   
{'loss': 0.2134, 'grad_norm': 0.07864632457494736, 'learning_rate': 0.00020714837136988075, 'epoch': 0.89}
+
 18%|█▊        | 134/750 [13:59<1:00:30,  5.89s/it]
 18%|█▊        | 135/750 [14:05<1:00:04,  5.86s/it]
                                                   
{'loss': 0.2359, 'grad_norm': 0.08450440317392349, 'learning_rate': 0.00020637972912229674, 'epoch': 0.9}
+
 18%|█▊        | 135/750 [14:05<1:00:04,  5.86s/it]
 18%|█▊        | 136/750 [14:10<59:52,  5.85s/it]  
                                                 
{'loss': 0.2258, 'grad_norm': 0.08534865081310272, 'learning_rate': 0.00020561958020749358, 'epoch': 0.91}
+
 18%|█▊        | 136/750 [14:10<59:52,  5.85s/it]
 18%|█▊        | 137/750 [14:16<1:00:12,  5.89s/it]
                                                   
{'loss': 0.2199, 'grad_norm': 0.07890835404396057, 'learning_rate': 0.00020486776935308305, 'epoch': 0.91}
+
 18%|█▊        | 137/750 [14:16<1:00:12,  5.89s/it]
 18%|█▊        | 138/750 [14:22<1:00:18,  5.91s/it]
                                                   
{'loss': 0.2236, 'grad_norm': 0.07853742688894272, 'learning_rate': 0.00020412414523193154, 'epoch': 0.92}
+
 18%|█▊        | 138/750 [14:22<1:00:18,  5.91s/it]
 19%|█▊        | 139/750 [14:28<1:00:13,  5.91s/it]
                                                   
{'loss': 0.2238, 'grad_norm': 0.080318383872509, 'learning_rate': 0.00020338856033420402, 'epoch': 0.93}
+
 19%|█▊        | 139/750 [14:28<1:00:13,  5.91s/it]
 19%|█▊        | 140/750 [14:34<59:18,  5.83s/it]  
                                                 
{'loss': 0.2135, 'grad_norm': 0.08497676998376846, 'learning_rate': 0.00020266087084444437, 'epoch': 0.93}
+
 19%|█▊        | 140/750 [14:34<59:18,  5.83s/it]
 19%|█▉        | 141/750 [14:40<59:25,  5.85s/it]
                                                 
{'loss': 0.2088, 'grad_norm': 0.07876597344875336, 'learning_rate': 0.00020194093652345886, 'epoch': 0.94}
+
 19%|█▉        | 141/750 [14:40<59:25,  5.85s/it]
 19%|█▉        | 142/750 [14:46<59:32,  5.88s/it]
                                                 
{'loss': 0.2065, 'grad_norm': 0.08852185308933258, 'learning_rate': 0.00020122862059478237, 'epoch': 0.95}
+
 19%|█▉        | 142/750 [14:46<59:32,  5.88s/it]
 19%|█▉        | 143/750 [14:52<59:38,  5.90s/it]
                                                 
{'loss': 0.2276, 'grad_norm': 0.07818302512168884, 'learning_rate': 0.00020052378963551984, 'epoch': 0.95}
+
 19%|█▉        | 143/750 [14:52<59:38,  5.90s/it]
 19%|█▉        | 144/750 [14:58<59:32,  5.90s/it]
                                                 
{'loss': 0.2184, 'grad_norm': 0.07803085446357727, 'learning_rate': 0.00019982631347136332, 'epoch': 0.96}
+
 19%|█▉        | 144/750 [14:58<59:32,  5.90s/it]
 19%|█▉        | 145/750 [15:04<59:45,  5.93s/it]
                                                 
{'loss': 0.2136, 'grad_norm': 0.07864702492952347, 'learning_rate': 0.00019913606507559878, 'epoch': 0.97}
+
 19%|█▉        | 145/750 [15:04<59:45,  5.93s/it]
 19%|█▉        | 146/750 [15:10<59:43,  5.93s/it]
                                                 
{'loss': 0.215, 'grad_norm': 0.07749994099140167, 'learning_rate': 0.00019845292047192357, 'epoch': 0.97}
+
 19%|█▉        | 146/750 [15:10<59:43,  5.93s/it]
 20%|█▉        | 147/750 [15:16<1:00:04,  5.98s/it]
                                                   
{'loss': 0.2099, 'grad_norm': 0.07554192841053009, 'learning_rate': 0.00019777675864090656, 'epoch': 0.98}
+
 20%|█▉        | 147/750 [15:16<1:00:04,  5.98s/it]
 20%|█▉        | 148/750 [15:22<59:34,  5.94s/it]  
                                                 
{'loss': 0.2317, 'grad_norm': 0.07940760999917984, 'learning_rate': 0.00019710746142993, 'epoch': 0.99}
+
 20%|█▉        | 148/750 [15:22<59:34,  5.94s/it]
 20%|█▉        | 149/750 [15:27<59:32,  5.94s/it]
                                                 
{'loss': 0.2115, 'grad_norm': 0.08441729098558426, 'learning_rate': 0.00019644491346646172, 'epoch': 0.99}
+
 20%|█▉        | 149/750 [15:27<59:32,  5.94s/it]
 20%|██        | 150/750 [15:36<1:05:51,  6.59s/it]
                                                   
{'loss': 0.2453, 'grad_norm': 0.08605577796697617, 'learning_rate': 0.00019578900207451217, 'epoch': 1.0}
+
 20%|██        | 150/750 [15:36<1:05:51,  6.59s/it]
 20%|██        | 151/750 [15:50<1:29:27,  8.96s/it]
                                                   
{'loss': 0.1873, 'grad_norm': 0.07326822727918625, 'learning_rate': 0.00019513961719414, 'epoch': 1.01}
+
 20%|██        | 151/750 [15:50<1:29:27,  8.96s/it]
 20%|██        | 152/750 [15:56<1:20:19,  8.06s/it]
                                                   
{'loss': 0.1967, 'grad_norm': 0.07358783483505249, 'learning_rate': 0.00019449665130387477, 'epoch': 1.01}
+
 20%|██        | 152/750 [15:56<1:20:19,  8.06s/it]
 20%|██        | 153/750 [16:02<1:13:25,  7.38s/it]
                                                   
{'loss': 0.1956, 'grad_norm': 0.07992872595787048, 'learning_rate': 0.0001938599993459332, 'epoch': 1.02}
+
 20%|██        | 153/750 [16:02<1:13:25,  7.38s/it]
 21%|██        | 154/750 [16:08<1:08:51,  6.93s/it]
                                                   
{'loss': 0.2365, 'grad_norm': 0.08515249937772751, 'learning_rate': 0.00019322955865411052, 'epoch': 1.03}
+
 21%|██        | 154/750 [16:08<1:08:51,  6.93s/it]
 21%|██        | 155/750 [16:14<1:05:55,  6.65s/it]
                                                   
{'loss': 0.1906, 'grad_norm': 0.09436432272195816, 'learning_rate': 0.00019260522888423458, 'epoch': 1.03}
+
 21%|██        | 155/750 [16:14<1:05:55,  6.65s/it]
 21%|██        | 156/750 [16:20<1:03:44,  6.44s/it]
                                                   
{'loss': 0.2096, 'grad_norm': 0.08241938054561615, 'learning_rate': 0.00019198691194707612, 'epoch': 1.04}
+
 21%|██        | 156/750 [16:20<1:03:44,  6.44s/it]
 21%|██        | 157/750 [16:25<1:01:52,  6.26s/it]
                                                   
{'loss': 0.2026, 'grad_norm': 0.08507107943296432, 'learning_rate': 0.00019137451194361295, 'epoch': 1.05}
+
 21%|██        | 157/750 [16:26<1:01:52,  6.26s/it]
 21%|██        | 158/750 [16:31<1:00:37,  6.14s/it]
                                                   
{'loss': 0.1888, 'grad_norm': 0.08608890324831009, 'learning_rate': 0.00019076793510255105, 'epoch': 1.05}
+
 21%|██        | 158/750 [16:31<1:00:37,  6.14s/it]
 21%|██        | 159/750 [16:37<59:57,  6.09s/it]  
                                                 
{'loss': 0.2272, 'grad_norm': 0.09399877488613129, 'learning_rate': 0.00019016708972001076, 'epoch': 1.06}
+
 21%|██        | 159/750 [16:37<59:57,  6.09s/it]
 21%|██▏       | 160/750 [16:43<59:39,  6.07s/it]
                                                 
{'loss': 0.2205, 'grad_norm': 0.08573122322559357, 'learning_rate': 0.00018957188610128878, 'epoch': 1.07}
+
 21%|██▏       | 160/750 [16:43<59:39,  6.07s/it]
 21%|██▏       | 161/750 [16:49<59:12,  6.03s/it]
                                                 
{'loss': 0.1932, 'grad_norm': 0.08784767985343933, 'learning_rate': 0.0001889822365046136, 'epoch': 1.07}
+
 21%|██▏       | 161/750 [16:49<59:12,  6.03s/it]
 22%|██▏       | 162/750 [16:55<58:30,  5.97s/it]
                                                 
{'loss': 0.2048, 'grad_norm': 0.08680360019207001, 'learning_rate': 0.00018839805508681298, 'epoch': 1.08}
+
 22%|██▏       | 162/750 [16:55<58:30,  5.97s/it]
 22%|██▏       | 163/750 [17:01<58:00,  5.93s/it]
                                                 
{'loss': 0.2107, 'grad_norm': 0.08761437237262726, 'learning_rate': 0.00018781925785081761, 'epoch': 1.09}
+
 22%|██▏       | 163/750 [17:01<58:00,  5.93s/it]
 22%|██▏       | 164/750 [17:07<57:52,  5.93s/it]
                                                 
{'loss': 0.2051, 'grad_norm': 0.08605002611875534, 'learning_rate': 0.0001872457625949279, 'epoch': 1.09}
+
 22%|██▏       | 164/750 [17:07<57:52,  5.93s/it]
 22%|██▏       | 165/750 [17:13<57:53,  5.94s/it]
                                                 
{'loss': 0.2063, 'grad_norm': 0.08843623846769333, 'learning_rate': 0.00018667748886377504, 'epoch': 1.1}
+
 22%|██▏       | 165/750 [17:13<57:53,  5.94s/it]
 22%|██▏       | 166/750 [17:19<57:34,  5.91s/it]
                                                 
{'loss': 0.1889, 'grad_norm': 0.0773942843079567, 'learning_rate': 0.00018611435790090854, 'epoch': 1.11}
+
 22%|██▏       | 166/750 [17:19<57:34,  5.91s/it]
 22%|██▏       | 167/750 [17:25<58:20,  6.01s/it]
                                                 
{'loss': 0.2079, 'grad_norm': 0.08796320110559464, 'learning_rate': 0.00018555629260294813, 'epoch': 1.11}
+
 22%|██▏       | 167/750 [17:25<58:20,  6.01s/it]
 22%|██▏       | 168/750 [17:31<58:03,  5.99s/it]
                                                 
{'loss': 0.1992, 'grad_norm': 0.08784277737140656, 'learning_rate': 0.00018500321747523874, 'epoch': 1.12}
+
 22%|██▏       | 168/750 [17:31<58:03,  5.99s/it]
 23%|██▎       | 169/750 [17:37<57:40,  5.96s/it]
                                                 
{'loss': 0.1875, 'grad_norm': 0.08896654099225998, 'learning_rate': 0.00018445505858895076, 'epoch': 1.13}
+
 23%|██▎       | 169/750 [17:37<57:40,  5.96s/it]
 23%|██▎       | 170/750 [17:43<57:23,  5.94s/it]
                                                 
{'loss': 0.2253, 'grad_norm': 0.09673389792442322, 'learning_rate': 0.0001839117435395704, 'epoch': 1.13}
+
 23%|██▎       | 170/750 [17:43<57:23,  5.94s/it]
 23%|██▎       | 171/750 [17:49<57:23,  5.95s/it]
                                                 
{'loss': 0.193, 'grad_norm': 0.08884169161319733, 'learning_rate': 0.00018337320140672696, 'epoch': 1.14}
+
 23%|██▎       | 171/750 [17:49<57:23,  5.95s/it]
 23%|██▎       | 172/750 [17:55<57:43,  5.99s/it]
                                                 
{'loss': 0.2156, 'grad_norm': 0.09978416562080383, 'learning_rate': 0.00018283936271530683, 'epoch': 1.15}
+
 23%|██▎       | 172/750 [17:55<57:43,  5.99s/it]
 23%|██▎       | 173/750 [18:01<57:34,  5.99s/it]
                                                 
{'loss': 0.2188, 'grad_norm': 0.0941423550248146, 'learning_rate': 0.0001823101593978053, 'epoch': 1.15}
+
 23%|██▎       | 173/750 [18:01<57:34,  5.99s/it]
 23%|██▎       | 174/750 [18:07<57:16,  5.97s/it]
                                                 
{'loss': 0.1908, 'grad_norm': 0.09383272379636765, 'learning_rate': 0.00018178552475787025, 'epoch': 1.16}
+
 23%|██▎       | 174/750 [18:07<57:16,  5.97s/it]
 23%|██▎       | 175/750 [18:13<57:14,  5.97s/it]
                                                 
{'loss': 0.2174, 'grad_norm': 0.09352857619524002, 'learning_rate': 0.00018126539343499314, 'epoch': 1.17}
+
 23%|██▎       | 175/750 [18:13<57:14,  5.97s/it]
 23%|██▎       | 176/750 [18:19<57:11,  5.98s/it]
                                                 
{'loss': 0.1764, 'grad_norm': 0.09135160595178604, 'learning_rate': 0.0001807497013703053, 'epoch': 1.17}
+
 23%|██▎       | 176/750 [18:19<57:11,  5.98s/it]
 24%|██▎       | 177/750 [18:24<56:42,  5.94s/it]
                                                 
{'loss': 0.1996, 'grad_norm': 0.09139572829008102, 'learning_rate': 0.00018023838577343806, 'epoch': 1.18}
+
 24%|██▎       | 177/750 [18:24<56:42,  5.94s/it]
 24%|██▎       | 178/750 [18:30<56:39,  5.94s/it]
                                                 
{'loss': 0.1999, 'grad_norm': 0.10222291946411133, 'learning_rate': 0.0001797313850904089, 'epoch': 1.19}
+
 24%|██▎       | 178/750 [18:30<56:39,  5.94s/it]
 24%|██▍       | 179/750 [18:36<56:31,  5.94s/it]
                                                 
{'loss': 0.2086, 'grad_norm': 0.09236767888069153, 'learning_rate': 0.00017922863897249503, 'epoch': 1.19}
+
 24%|██▍       | 179/750 [18:36<56:31,  5.94s/it]
 24%|██▍       | 180/750 [18:42<56:23,  5.94s/it]
                                                 
{'loss': 0.1913, 'grad_norm': 0.09252063930034637, 'learning_rate': 0.00017873008824606016, 'epoch': 1.2}
+
 24%|██▍       | 180/750 [18:42<56:23,  5.94s/it]
 24%|██▍       | 181/750 [18:48<56:04,  5.91s/it]
                                                 
{'loss': 0.1966, 'grad_norm': 0.09938914328813553, 'learning_rate': 0.00017823567488329874, 'epoch': 1.21}
+
 24%|██▍       | 181/750 [18:48<56:04,  5.91s/it]
 24%|██▍       | 182/750 [18:54<56:00,  5.92s/it]
                                                 
{'loss': 0.1928, 'grad_norm': 0.09783317893743515, 'learning_rate': 0.00017774534197386607, 'epoch': 1.21}
+
 24%|██▍       | 182/750 [18:54<56:00,  5.92s/it]
 24%|██▍       | 183/750 [19:00<55:52,  5.91s/it]
                                                 
{'loss': 0.1888, 'grad_norm': 0.09500114619731903, 'learning_rate': 0.0001772590336973617, 'epoch': 1.22}
+
 24%|██▍       | 183/750 [19:00<55:52,  5.91s/it]
 25%|██▍       | 184/750 [19:06<55:40,  5.90s/it]
                                                 
{'loss': 0.2, 'grad_norm': 0.09481227397918701, 'learning_rate': 0.00017677669529663688, 'epoch': 1.23}
+
 25%|██▍       | 184/750 [19:06<55:40,  5.90s/it]
 25%|██▍       | 185/750 [19:12<55:51,  5.93s/it]
                                                 
{'loss': 0.1879, 'grad_norm': 0.10236822813749313, 'learning_rate': 0.00017629827305189658, 'epoch': 1.23}
+
 25%|██▍       | 185/750 [19:12<55:51,  5.93s/it]
 25%|██▍       | 186/750 [19:18<56:24,  6.00s/it]
                                                 
{'loss': 0.1977, 'grad_norm': 0.10663288086652756, 'learning_rate': 0.0001758237142555681, 'epoch': 1.24}
+
 25%|██▍       | 186/750 [19:18<56:24,  6.00s/it]
 25%|██▍       | 187/750 [19:24<56:02,  5.97s/it]
                                                 
{'loss': 0.2222, 'grad_norm': 0.10987573862075806, 'learning_rate': 0.00017535296718791012, 'epoch': 1.25}
+
 25%|██▍       | 187/750 [19:24<56:02,  5.97s/it]
 25%|██▌       | 188/750 [19:30<55:35,  5.93s/it]
                                                 
{'loss': 0.2164, 'grad_norm': 0.10460679978132248, 'learning_rate': 0.0001748859810933361, 'epoch': 1.25}
+
 25%|██▌       | 188/750 [19:30<55:35,  5.93s/it]
 25%|██▌       | 189/750 [19:36<55:34,  5.94s/it]
                                                 
{'loss': 0.2004, 'grad_norm': 0.10028848052024841, 'learning_rate': 0.00017442270615742787, 'epoch': 1.26}
+
 25%|██▌       | 189/750 [19:36<55:34,  5.94s/it]
 25%|██▌       | 190/750 [19:42<55:22,  5.93s/it]
                                                 
{'loss': 0.1987, 'grad_norm': 0.10360311716794968, 'learning_rate': 0.00017396309348461487, 'epoch': 1.27}
+
 25%|██▌       | 190/750 [19:42<55:22,  5.93s/it]
 25%|██▌       | 191/750 [19:48<55:15,  5.93s/it]
                                                 
{'loss': 0.1963, 'grad_norm': 0.10013779252767563, 'learning_rate': 0.00017350709507649772, 'epoch': 1.27}
+
 25%|██▌       | 191/750 [19:48<55:15,  5.93s/it]
 26%|██▌       | 192/750 [19:53<55:07,  5.93s/it]
                                                 
{'loss': 0.2058, 'grad_norm': 0.09892645478248596, 'learning_rate': 0.00017305466381079324, 'epoch': 1.28}
+
 26%|██▌       | 192/750 [19:53<55:07,  5.93s/it]
 26%|██▌       | 193/750 [19:59<54:51,  5.91s/it]
                                                 
{'loss': 0.2051, 'grad_norm': 0.09642582386732101, 'learning_rate': 0.00017260575342088004, 'epoch': 1.29}
+
 26%|██▌       | 193/750 [19:59<54:51,  5.91s/it]
 26%|██▌       | 194/750 [20:05<54:55,  5.93s/it]
                                                 
{'loss': 0.1975, 'grad_norm': 0.09456117451190948, 'learning_rate': 0.0001721603184759251, 'epoch': 1.29}
+
 26%|██▌       | 194/750 [20:05<54:55,  5.93s/it]
 26%|██▌       | 195/750 [20:11<55:01,  5.95s/it]
                                                 
{'loss': 0.1957, 'grad_norm': 0.10241963714361191, 'learning_rate': 0.0001717183143615715, 'epoch': 1.3}
+
 26%|██▌       | 195/750 [20:11<55:01,  5.95s/it]
 26%|██▌       | 196/750 [20:17<54:59,  5.96s/it]
                                                 
{'loss': 0.1576, 'grad_norm': 0.08325479179620743, 'learning_rate': 0.00017127969726116855, 'epoch': 1.31}
+
 26%|██▌       | 196/750 [20:17<54:59,  5.96s/it]
 26%|██▋       | 197/750 [20:23<55:02,  5.97s/it]
                                                 
{'loss': 0.2034, 'grad_norm': 0.09574084728956223, 'learning_rate': 0.0001708444241375271, 'epoch': 1.31}
+
 26%|██▋       | 197/750 [20:23<55:02,  5.97s/it]
 26%|██▋       | 198/750 [20:29<55:01,  5.98s/it]
                                                 
{'loss': 0.2132, 'grad_norm': 0.09876696765422821, 'learning_rate': 0.00017041245271518172, 'epoch': 1.32}
+
 26%|██▋       | 198/750 [20:29<55:01,  5.98s/it]
 27%|██▋       | 199/750 [20:35<54:49,  5.97s/it]
                                                 
{'loss': 0.1851, 'grad_norm': 0.09725328534841537, 'learning_rate': 0.00016998374146314418, 'epoch': 1.33}
+
 27%|██▋       | 199/750 [20:35<54:49,  5.97s/it]
 27%|██▋       | 200/750 [20:41<53:50,  5.87s/it]
                                                 
{'loss': 0.2032, 'grad_norm': 0.10103516280651093, 'learning_rate': 0.0001695582495781317, 'epoch': 1.33}
+
 27%|██▋       | 200/750 [20:41<53:50,  5.87s/it][INFO|trainer.py:3831] 2025-06-27 00:09:28,330 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-27 00:09:28,330 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-27 00:09:28,330 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:01<00:07,  1.07it/s]
+
 30%|███       | 3/10 [00:03<00:09,  1.30s/it]
+
 40%|████      | 4/10 [00:05<00:08,  1.50s/it]
+
 50%|█████     | 5/10 [00:07<00:08,  1.62s/it]
+
 60%|██████    | 6/10 [00:09<00:06,  1.69s/it]
+
 70%|███████   | 7/10 [00:11<00:05,  1.74s/it]
+
 80%|████████  | 8/10 [00:12<00:03,  1.76s/it]
+
 90%|█████████ | 9/10 [00:14<00:01,  1.77s/it]
+
100%|██████████| 10/10 [00:18<00:00,  2.46s/it]
                                                 
+
                                               
{'eval_loss': 0.21198447048664093, 'eval_accuracy': 0.22277381533952126, 'eval_runtime': 34.3769, 'eval_samples_per_second': 29.089, 'eval_steps_per_second': 0.291, 'epoch': 1.33}
+
 27%|██▋       | 200/750 [21:15<53:50,  5.87s/it]
+
100%|██████████| 10/10 [00:24<00:00,  2.46s/it]
+
                                               [INFO|trainer.py:3515] 2025-06-27 00:10:12,597 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-200
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d940c-6714583f6819fba559eba40a;6ea89e93-e0ba-4350-8087-8c66ff135abc)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-27 00:10:12,935 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-200/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-27 00:10:12,935 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-200/special_tokens_map.json
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[2025-06-27 00:10:17,953] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Checkpoint global_step200 is begin to save!
+[2025-06-27 00:10:17,976] [INFO] [logging.py:107:log_dist] [Rank 0] Saving model checkpoint: ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-200/global_step200/mp_rank_00_model_states.pt
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d9412-0525cb24664bc6a817a5199e;b69187b8-6c13-4836-9e8f-444b992560b1)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d9412-79eea90b0080f36458fd28e4;c5330cdc-d9ad-4653-a60c-e9792beac3da)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d9412-673b11620f41494e0798129d;1d3ca5be-5320-4237-a9ef-32d90bdc691b)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d9412-47a335377afb587169c716ce;b53874b3-dd19-4254-8815-5a03032f26b0)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+
 27%|██▋       | 201/750 [21:37<3:12:07, 21.00s/it]
                                                   
{'loss': 0.2006, 'grad_norm': 0.09802627563476562, 'learning_rate': 0.0001691359369682545, 'epoch': 1.34}
+
 27%|██▋       | 201/750 [21:37<3:12:07, 21.00s/it]
 27%|██▋       | 202/750 [21:43<2:30:12, 16.45s/it]
                                                   
{'loss': 0.187, 'grad_norm': 0.09598495066165924, 'learning_rate': 0.00016871676423714827, 'epoch': 1.35}
+
 27%|██▋       | 202/750 [21:43<2:30:12, 16.45s/it]
 27%|██▋       | 203/750 [21:49<2:01:51, 13.37s/it]
                                                   
{'loss': 0.2075, 'grad_norm': 0.10226590186357498, 'learning_rate': 0.00016830069266853705, 'epoch': 1.35}
+
 27%|██▋       | 203/750 [21:49<2:01:51, 13.37s/it]
 27%|██▋       | 204/750 [21:55<1:41:19, 11.13s/it]
                                                   
{'loss': 0.2012, 'grad_norm': 0.0975937768816948, 'learning_rate': 0.00016788768421121283, 'epoch': 1.36}
+
 27%|██▋       | 204/750 [21:55<1:41:19, 11.13s/it]
 27%|██▋       | 205/750 [22:01<1:27:20,  9.62s/it]
                                                   
{'loss': 0.1976, 'grad_norm': 0.09934520721435547, 'learning_rate': 0.00016747770146441848, 'epoch': 1.37}
+
 27%|██▋       | 205/750 [22:01<1:27:20,  9.62s/it]
 27%|██▋       | 206/750 [22:07<1:17:10,  8.51s/it]
                                                   
{'loss': 0.1941, 'grad_norm': 0.09936224669218063, 'learning_rate': 0.0001670707076636216, 'epoch': 1.37}
+
 27%|██▋       | 206/750 [22:07<1:17:10,  8.51s/it]
 28%|██▊       | 207/750 [22:13<1:09:45,  7.71s/it]
                                                   
{'loss': 0.1997, 'grad_norm': 0.10343389958143234, 'learning_rate': 0.00016666666666666666, 'epoch': 1.38}
+
 28%|██▊       | 207/750 [22:13<1:09:45,  7.71s/it]
 28%|██▊       | 208/750 [22:19<1:04:54,  7.18s/it]
                                                   
{'loss': 0.1948, 'grad_norm': 0.09763069450855255, 'learning_rate': 0.0001662655429402941, 'epoch': 1.39}
+
 28%|██▊       | 208/750 [22:19<1:04:54,  7.18s/it]
 28%|██▊       | 209/750 [22:25<1:01:35,  6.83s/it]
                                                   
{'loss': 0.19, 'grad_norm': 0.08771580457687378, 'learning_rate': 0.00016586730154701388, 'epoch': 1.39}
+
 28%|██▊       | 209/750 [22:25<1:01:35,  6.83s/it]
 28%|██▊       | 210/750 [22:31<59:06,  6.57s/it]  
                                                 
{'loss': 0.1954, 'grad_norm': 0.10181500762701035, 'learning_rate': 0.00016547190813232432, 'epoch': 1.4}
+
 28%|██▊       | 210/750 [22:31<59:06,  6.57s/it]
 28%|██▊       | 211/750 [22:37<57:33,  6.41s/it]
                                                 
{'loss': 0.1923, 'grad_norm': 0.0967947468161583, 'learning_rate': 0.00016507932891226336, 'epoch': 1.41}
+
 28%|██▊       | 211/750 [22:37<57:33,  6.41s/it]
 28%|██▊       | 212/750 [22:43<55:56,  6.24s/it]
                                                 
{'loss': 0.1963, 'grad_norm': 0.09669908881187439, 'learning_rate': 0.00016468953066128386, 'epoch': 1.41}
+
 28%|██▊       | 212/750 [22:43<55:56,  6.24s/it]
 28%|██▊       | 213/750 [22:49<55:14,  6.17s/it]
                                                 
{'loss': 0.2168, 'grad_norm': 0.10868806391954422, 'learning_rate': 0.00016430248070044244, 'epoch': 1.42}
+
 28%|██▊       | 213/750 [22:49<55:14,  6.17s/it]
 29%|██▊       | 214/750 [22:55<54:32,  6.10s/it]
                                                 
{'loss': 0.2249, 'grad_norm': 0.10268723219633102, 'learning_rate': 0.0001639181468858914, 'epoch': 1.43}
+
 29%|██▊       | 214/750 [22:55<54:32,  6.10s/it]
 29%|██▊       | 215/750 [23:01<54:22,  6.10s/it]
                                                 
{'loss': 0.1849, 'grad_norm': 0.09193839132785797, 'learning_rate': 0.00016353649759766664, 'epoch': 1.43}
+
 29%|██▊       | 215/750 [23:01<54:22,  6.10s/it]
 29%|██▉       | 216/750 [23:07<54:06,  6.08s/it]
                                                 
{'loss': 0.1821, 'grad_norm': 0.09557466208934784, 'learning_rate': 0.00016315750172876014, 'epoch': 1.44}
+
 29%|██▉       | 216/750 [23:07<54:06,  6.08s/it]
 29%|██▉       | 217/750 [23:13<53:50,  6.06s/it]
                                                 
{'loss': 0.1937, 'grad_norm': 0.09658323973417282, 'learning_rate': 0.00016278112867447063, 'epoch': 1.45}
+
 29%|██▉       | 217/750 [23:13<53:50,  6.06s/it]
 29%|██▉       | 218/750 [23:19<53:15,  6.01s/it]
                                                 
{'loss': 0.1884, 'grad_norm': 0.09883001446723938, 'learning_rate': 0.00016240734832202275, 'epoch': 1.45}
+
 29%|██▉       | 218/750 [23:19<53:15,  6.01s/it]
 29%|██▉       | 219/750 [23:24<51:57,  5.87s/it]
                                                 
{'loss': 0.1738, 'grad_norm': 0.09773606806993484, 'learning_rate': 0.00016203613104044751, 'epoch': 1.46}
+
 29%|██▉       | 219/750 [23:24<51:57,  5.87s/it]
 29%|██▉       | 220/750 [23:30<52:14,  5.91s/it]
                                                 
{'loss': 0.1874, 'grad_norm': 0.09291878342628479, 'learning_rate': 0.00016166744767071581, 'epoch': 1.47}
+
 29%|██▉       | 220/750 [23:30<52:14,  5.91s/it]
 29%|██▉       | 221/750 [23:36<51:34,  5.85s/it]
                                                 
{'loss': 0.1992, 'grad_norm': 0.11277790367603302, 'learning_rate': 0.00016130126951611793, 'epoch': 1.47}
+
 29%|██▉       | 221/750 [23:36<51:34,  5.85s/it]
 30%|██▉       | 222/750 [23:42<52:28,  5.96s/it]
                                                 
{'loss': 0.1929, 'grad_norm': 0.1059861108660698, 'learning_rate': 0.0001609375683328815, 'epoch': 1.48}
+
 30%|██▉       | 222/750 [23:42<52:28,  5.96s/it]
 30%|██▉       | 223/750 [23:48<52:27,  5.97s/it]
                                                 
{'loss': 0.2077, 'grad_norm': 0.0982198640704155, 'learning_rate': 0.00016057631632102133, 'epoch': 1.49}
+
 30%|██▉       | 223/750 [23:48<52:27,  5.97s/it]
 30%|██▉       | 224/750 [23:54<52:08,  5.95s/it]
                                                 
{'loss': 0.1968, 'grad_norm': 0.10334406793117523, 'learning_rate': 0.00016021748611541394, 'epoch': 1.49}
+
 30%|██▉       | 224/750 [23:54<52:08,  5.95s/it]
 30%|███       | 225/750 [24:00<52:08,  5.96s/it]
                                                 
{'loss': 0.1918, 'grad_norm': 0.09775006771087646, 'learning_rate': 0.00015986105077709064, 'epoch': 1.5}
+
 30%|███       | 225/750 [24:00<52:08,  5.96s/it]
 30%|███       | 226/750 [24:06<52:11,  5.98s/it]
                                                 
{'loss': 0.2041, 'grad_norm': 0.09927330166101456, 'learning_rate': 0.00015950698378474278, 'epoch': 1.51}
+
 30%|███       | 226/750 [24:06<52:11,  5.98s/it]
 30%|███       | 227/750 [24:12<52:08,  5.98s/it]
                                                 
{'loss': 0.1932, 'grad_norm': 0.09958826750516891, 'learning_rate': 0.00015915525902643283, 'epoch': 1.51}
+
 30%|███       | 227/750 [24:12<52:08,  5.98s/it]
 30%|███       | 228/750 [24:18<51:38,  5.94s/it]
                                                 
{'loss': 0.1951, 'grad_norm': 0.10214662551879883, 'learning_rate': 0.0001588058507915059, 'epoch': 1.52}
+
 30%|███       | 228/750 [24:18<51:38,  5.94s/it]
 31%|███       | 229/750 [24:24<51:23,  5.92s/it]
                                                 
{'loss': 0.192, 'grad_norm': 0.1043030396103859, 'learning_rate': 0.00015845873376269562, 'epoch': 1.53}
+
 31%|███       | 229/750 [24:24<51:23,  5.92s/it]
 31%|███       | 230/750 [24:30<51:12,  5.91s/it]
                                                 
{'loss': 0.2051, 'grad_norm': 0.10158228129148483, 'learning_rate': 0.00015811388300841897, 'epoch': 1.53}
+
 31%|███       | 230/750 [24:30<51:12,  5.91s/it]
 31%|███       | 231/750 [24:35<50:52,  5.88s/it]
                                                 
{'loss': 0.2134, 'grad_norm': 0.10783704370260239, 'learning_rate': 0.00015777127397525472, 'epoch': 1.54}
+
 31%|███       | 231/750 [24:35<50:52,  5.88s/it]
 31%|███       | 232/750 [24:41<51:07,  5.92s/it]
                                                 
{'loss': 0.2005, 'grad_norm': 0.10326651483774185, 'learning_rate': 0.00015743088248060063, 'epoch': 1.55}
+
 31%|███       | 232/750 [24:41<51:07,  5.92s/it]
 31%|███       | 233/750 [24:47<51:02,  5.92s/it]
                                                 
{'loss': 0.2053, 'grad_norm': 0.1080598458647728, 'learning_rate': 0.0001570926847055038, 'epoch': 1.55}
+
 31%|███       | 233/750 [24:47<51:02,  5.92s/it]
 31%|███       | 234/750 [24:53<51:06,  5.94s/it]
                                                 
{'loss': 0.1944, 'grad_norm': 0.09971864521503448, 'learning_rate': 0.00015675665718766006, 'epoch': 1.56}
+
 31%|███       | 234/750 [24:53<51:06,  5.94s/it]
 31%|███▏      | 235/750 [24:59<50:55,  5.93s/it]
                                                 
{'loss': 0.2076, 'grad_norm': 0.10496847331523895, 'learning_rate': 0.00015642277681457702, 'epoch': 1.57}
+
 31%|███▏      | 235/750 [24:59<50:55,  5.93s/it]
 31%|███▏      | 236/750 [25:05<50:45,  5.93s/it]
                                                 
{'loss': 0.2104, 'grad_norm': 0.09720281511545181, 'learning_rate': 0.00015609102081689716, 'epoch': 1.57}
+
 31%|███▏      | 236/750 [25:05<50:45,  5.93s/it]
 32%|███▏      | 237/750 [25:11<50:31,  5.91s/it]
                                                 
{'loss': 0.1897, 'grad_norm': 0.1059885248541832, 'learning_rate': 0.00015576136676187527, 'epoch': 1.58}
+
 32%|███▏      | 237/750 [25:11<50:31,  5.91s/it]
 32%|███▏      | 238/750 [25:17<50:24,  5.91s/it]
                                                 
{'loss': 0.2077, 'grad_norm': 0.10878182202577591, 'learning_rate': 0.0001554337925470077, 'epoch': 1.59}
+
 32%|███▏      | 238/750 [25:17<50:24,  5.91s/it]
 32%|███▏      | 239/750 [25:23<50:27,  5.92s/it]
                                                 
{'loss': 0.1978, 'grad_norm': 0.10656464844942093, 'learning_rate': 0.00015510827639380736, 'epoch': 1.59}
+
 32%|███▏      | 239/750 [25:23<50:27,  5.92s/it]
 32%|███▏      | 240/750 [25:29<50:32,  5.95s/it]
                                                 
{'loss': 0.2059, 'grad_norm': 0.09944751858711243, 'learning_rate': 0.00015478479684172258, 'epoch': 1.6}
+
 32%|███▏      | 240/750 [25:29<50:32,  5.95s/it]
 32%|███▏      | 241/750 [25:35<50:52,  6.00s/it]
                                                 
{'loss': 0.1813, 'grad_norm': 0.10171686112880707, 'learning_rate': 0.00015446333274219396, 'epoch': 1.61}
+
 32%|███▏      | 241/750 [25:35<50:52,  6.00s/it]
 32%|███▏      | 242/750 [25:41<50:50,  6.00s/it]
                                                 
{'loss': 0.2015, 'grad_norm': 0.10150379687547684, 'learning_rate': 0.000154143863252847, 'epoch': 1.61}
+
 32%|███▏      | 242/750 [25:41<50:50,  6.00s/it]
 32%|███▏      | 243/750 [25:47<50:20,  5.96s/it]
                                                 
{'loss': 0.19, 'grad_norm': 0.09964952617883682, 'learning_rate': 0.0001538263678318162, 'epoch': 1.62}
+
 32%|███▏      | 243/750 [25:47<50:20,  5.96s/it]
 33%|███▎      | 244/750 [25:53<50:14,  5.96s/it]
                                                 
{'loss': 0.2034, 'grad_norm': 0.10983213782310486, 'learning_rate': 0.00015351082623219707, 'epoch': 1.63}
+
 33%|███▎      | 244/750 [25:53<50:14,  5.96s/it]
 33%|███▎      | 245/750 [25:59<49:56,  5.93s/it]
                                                 
{'loss': 0.211, 'grad_norm': 0.10578607767820358, 'learning_rate': 0.00015319721849662297, 'epoch': 1.63}
+
 33%|███▎      | 245/750 [25:59<49:56,  5.93s/it]
 33%|███▎      | 246/750 [26:05<49:55,  5.94s/it]
                                                 
{'loss': 0.2101, 'grad_norm': 0.1035049706697464, 'learning_rate': 0.0001528855249519633, 'epoch': 1.64}
+
 33%|███▎      | 246/750 [26:05<49:55,  5.94s/it]
 33%|███▎      | 247/750 [26:11<49:56,  5.96s/it]
                                                 
{'loss': 0.2217, 'grad_norm': 0.09901197999715805, 'learning_rate': 0.00015257572620413985, 'epoch': 1.65}
+
 33%|███▎      | 247/750 [26:11<49:56,  5.96s/it]
 33%|███▎      | 248/750 [26:17<49:48,  5.95s/it]
                                                 
{'loss': 0.1959, 'grad_norm': 0.09957315027713776, 'learning_rate': 0.00015226780313305814, 'epoch': 1.65}
+
 33%|███▎      | 248/750 [26:17<49:48,  5.95s/it]
 33%|███▎      | 249/750 [26:23<49:47,  5.96s/it]
                                                 
{'loss': 0.1845, 'grad_norm': 0.10152138769626617, 'learning_rate': 0.00015196173688765093, 'epoch': 1.66}
+
 33%|███▎      | 249/750 [26:23<49:47,  5.96s/it]
 33%|███▎      | 250/750 [26:29<49:34,  5.95s/it]
                                                 
{'loss': 0.1907, 'grad_norm': 0.10356403142213821, 'learning_rate': 0.00015165750888103104, 'epoch': 1.67}
+
 33%|███▎      | 250/750 [26:29<49:34,  5.95s/it]
 33%|███▎      | 251/750 [26:34<49:13,  5.92s/it]
                                                 
{'loss': 0.1966, 'grad_norm': 0.10825596004724503, 'learning_rate': 0.00015135510078575002, 'epoch': 1.67}
+
 33%|███▎      | 251/750 [26:34<49:13,  5.92s/it]
 34%|███▎      | 252/750 [26:40<49:12,  5.93s/it]
                                                 
{'loss': 0.1849, 'grad_norm': 0.101904958486557, 'learning_rate': 0.00015105449452916094, 'epoch': 1.68}
+
 34%|███▎      | 252/750 [26:40<49:12,  5.93s/it]
 34%|███▎      | 253/750 [26:46<49:04,  5.93s/it]
                                                 
{'loss': 0.1913, 'grad_norm': 0.10389713197946548, 'learning_rate': 0.00015075567228888182, 'epoch': 1.69}
+
 34%|███▎      | 253/750 [26:46<49:04,  5.93s/it]
 34%|███▍      | 254/750 [26:52<48:57,  5.92s/it]
                                                 
{'loss': 0.2084, 'grad_norm': 0.10135425627231598, 'learning_rate': 0.0001504586164883572, 'epoch': 1.69}
+
 34%|███▍      | 254/750 [26:52<48:57,  5.92s/it]
 34%|███▍      | 255/750 [26:58<48:52,  5.92s/it]
                                                 
{'loss': 0.1956, 'grad_norm': 0.09966433793306351, 'learning_rate': 0.00015016330979251603, 'epoch': 1.7}
+
 34%|███▍      | 255/750 [26:58<48:52,  5.92s/it]
 34%|███▍      | 256/750 [27:04<48:42,  5.92s/it]
                                                 
{'loss': 0.2018, 'grad_norm': 0.10264011472463608, 'learning_rate': 0.00014986973510352247, 'epoch': 1.71}
+
 34%|███▍      | 256/750 [27:04<48:42,  5.92s/it]
 34%|███▍      | 257/750 [27:10<48:39,  5.92s/it]
                                                 
{'loss': 0.2023, 'grad_norm': 0.10318576544523239, 'learning_rate': 0.0001495778755566181, 'epoch': 1.71}
+
 34%|███▍      | 257/750 [27:10<48:39,  5.92s/it]
 34%|███▍      | 258/750 [27:16<48:25,  5.91s/it]
                                                 
{'loss': 0.1893, 'grad_norm': 0.10010098665952682, 'learning_rate': 0.0001492877145160524, 'epoch': 1.72}
+
 34%|███▍      | 258/750 [27:16<48:25,  5.91s/it]
 35%|███▍      | 259/750 [27:22<48:10,  5.89s/it]
                                                 
{'loss': 0.1984, 'grad_norm': 0.09545325487852097, 'learning_rate': 0.00014899923557110017, 'epoch': 1.73}
+
 35%|███▍      | 259/750 [27:22<48:10,  5.89s/it]
 35%|███▍      | 260/750 [27:28<48:34,  5.95s/it]
                                                 
{'loss': 0.2001, 'grad_norm': 0.09963326156139374, 'learning_rate': 0.0001487124225321631, 'epoch': 1.73}
+
 35%|███▍      | 260/750 [27:28<48:34,  5.95s/it]
 35%|███▍      | 261/750 [27:34<48:39,  5.97s/it]
                                                 
{'loss': 0.1866, 'grad_norm': 0.10346517711877823, 'learning_rate': 0.00014842725942695355, 'epoch': 1.74}
+
 35%|███▍      | 261/750 [27:34<48:39,  5.97s/it]
 35%|███▍      | 262/750 [27:40<48:22,  5.95s/it]
                                                 
{'loss': 0.1709, 'grad_norm': 0.1005755364894867, 'learning_rate': 0.0001481437304967584, 'epoch': 1.75}
+
 35%|███▍      | 262/750 [27:40<48:22,  5.95s/it]
 35%|███▌      | 263/750 [27:46<48:17,  5.95s/it]
                                                 
{'loss': 0.2003, 'grad_norm': 0.10314761102199554, 'learning_rate': 0.00014786182019278145, 'epoch': 1.75}
+
 35%|███▌      | 263/750 [27:46<48:17,  5.95s/it]
 35%|███▌      | 264/750 [27:52<48:06,  5.94s/it]
                                                 
{'loss': 0.1899, 'grad_norm': 0.10142087191343307, 'learning_rate': 0.0001475815131725618, 'epoch': 1.76}
+
 35%|███▌      | 264/750 [27:52<48:06,  5.94s/it]
 35%|███▌      | 265/750 [27:58<48:04,  5.95s/it]
                                                 
{'loss': 0.1996, 'grad_norm': 0.10087636858224869, 'learning_rate': 0.00014730279429646666, 'epoch': 1.77}
+
 35%|███▌      | 265/750 [27:58<48:04,  5.95s/it]
 35%|███▌      | 266/750 [28:03<47:49,  5.93s/it]
                                                 
{'loss': 0.197, 'grad_norm': 0.10521330684423447, 'learning_rate': 0.0001470256486242569, 'epoch': 1.77}
+
 35%|███▌      | 266/750 [28:03<47:49,  5.93s/it]
 36%|███▌      | 267/750 [28:09<47:36,  5.91s/it]
                                                 
{'loss': 0.192, 'grad_norm': 0.10543681681156158, 'learning_rate': 0.00014675006141172335, 'epoch': 1.78}
+
 36%|███▌      | 267/750 [28:09<47:36,  5.91s/it]
 36%|███▌      | 268/750 [28:15<47:32,  5.92s/it]
                                                 
{'loss': 0.2002, 'grad_norm': 0.10189599543809891, 'learning_rate': 0.00014647601810739195, 'epoch': 1.79}
+
 36%|███▌      | 268/750 [28:15<47:32,  5.92s/it]
 36%|███▌      | 269/750 [28:21<47:25,  5.92s/it]
                                                 
{'loss': 0.225, 'grad_norm': 0.11100862920284271, 'learning_rate': 0.0001462035043492969, 'epoch': 1.79}
+
 36%|███▌      | 269/750 [28:21<47:25,  5.92s/it]
 36%|███▌      | 270/750 [28:27<47:29,  5.94s/it]
                                                 
{'loss': 0.1919, 'grad_norm': 0.09958995133638382, 'learning_rate': 0.00014593250596181887, 'epoch': 1.8}
+
 36%|███▌      | 270/750 [28:27<47:29,  5.94s/it]
 36%|███▌      | 271/750 [28:33<47:16,  5.92s/it]
                                                 
{'loss': 0.1938, 'grad_norm': 0.10462497919797897, 'learning_rate': 0.00014566300895258813, 'epoch': 1.81}
+
 36%|███▌      | 271/750 [28:33<47:16,  5.92s/it]
 36%|███▋      | 272/750 [28:39<47:06,  5.91s/it]
                                                 
{'loss': 0.1929, 'grad_norm': 0.11180876940488815, 'learning_rate': 0.00014539499950944992, 'epoch': 1.81}
+
 36%|███▋      | 272/750 [28:39<47:06,  5.91s/it]
 36%|███▋      | 273/750 [28:45<47:10,  5.93s/it]
                                                 
{'loss': 0.1767, 'grad_norm': 0.0973215252161026, 'learning_rate': 0.00014512846399749107, 'epoch': 1.82}
+
 36%|███▋      | 273/750 [28:45<47:10,  5.93s/it]
 37%|███▋      | 274/750 [28:51<47:18,  5.96s/it]
                                                 
{'loss': 0.1796, 'grad_norm': 0.09500745683908463, 'learning_rate': 0.00014486338895612657, 'epoch': 1.83}
+
 37%|███▋      | 274/750 [28:51<47:18,  5.96s/it]
 37%|███▋      | 275/750 [28:57<46:55,  5.93s/it]
                                                 
{'loss': 0.1729, 'grad_norm': 0.10198456048965454, 'learning_rate': 0.00014459976109624422, 'epoch': 1.83}
+
 37%|███▋      | 275/750 [28:57<46:55,  5.93s/it]
 37%|███▋      | 276/750 [29:03<46:44,  5.92s/it]
                                                 
{'loss': 0.1964, 'grad_norm': 0.1011200100183487, 'learning_rate': 0.00014433756729740648, 'epoch': 1.84}
+
 37%|███▋      | 276/750 [29:03<46:44,  5.92s/it]
 37%|███▋      | 277/750 [29:08<46:24,  5.89s/it]
                                                 
{'loss': 0.2199, 'grad_norm': 0.11574704200029373, 'learning_rate': 0.00014407679460510741, 'epoch': 1.85}
+
 37%|███▋      | 277/750 [29:08<46:24,  5.89s/it]
 37%|███▋      | 278/750 [29:14<46:22,  5.89s/it]
                                                 
{'loss': 0.1884, 'grad_norm': 0.10165125131607056, 'learning_rate': 0.00014381743022808492, 'epoch': 1.85}
+
 37%|███▋      | 278/750 [29:14<46:22,  5.89s/it]
 37%|███▋      | 279/750 [29:20<46:31,  5.93s/it]
                                                 
{'loss': 0.1898, 'grad_norm': 0.09997145086526871, 'learning_rate': 0.00014355946153568483, 'epoch': 1.86}
+
 37%|███▋      | 279/750 [29:20<46:31,  5.93s/it]
 37%|███▋      | 280/750 [29:26<46:39,  5.96s/it]
                                                 
{'loss': 0.2153, 'grad_norm': 0.11056888848543167, 'learning_rate': 0.0001433028760552777, 'epoch': 1.87}
+
 37%|███▋      | 280/750 [29:26<46:39,  5.96s/it]
 37%|███▋      | 281/750 [29:32<46:19,  5.93s/it]
                                                 
{'loss': 0.2125, 'grad_norm': 0.10536083579063416, 'learning_rate': 0.00014304766146972588, 'epoch': 1.87}
+
 37%|███▋      | 281/750 [29:32<46:19,  5.93s/it]
 38%|███▊      | 282/750 [29:38<46:15,  5.93s/it]
                                                 
{'loss': 0.1987, 'grad_norm': 0.10255546122789383, 'learning_rate': 0.0001427938056148999, 'epoch': 1.88}
+
 38%|███▊      | 282/750 [29:38<46:15,  5.93s/it]
 38%|███▊      | 283/750 [29:44<46:06,  5.92s/it]
                                                 
{'loss': 0.1904, 'grad_norm': 0.10317458212375641, 'learning_rate': 0.00014254129647724352, 'epoch': 1.89}
+
 38%|███▊      | 283/750 [29:44<46:06,  5.92s/it]
 38%|███▊      | 284/750 [29:50<46:01,  5.93s/it]
                                                 
{'loss': 0.2045, 'grad_norm': 0.1047426164150238, 'learning_rate': 0.00014229012219138556, 'epoch': 1.89}
+
 38%|███▊      | 284/750 [29:50<46:01,  5.93s/it]
 38%|███▊      | 285/750 [29:56<45:52,  5.92s/it]
                                                 
{'loss': 0.2055, 'grad_norm': 0.09543642401695251, 'learning_rate': 0.00014204027103779837, 'epoch': 1.9}
+
 38%|███▊      | 285/750 [29:56<45:52,  5.92s/it]
 38%|███▊      | 286/750 [30:02<45:53,  5.93s/it]
                                                 
{'loss': 0.2, 'grad_norm': 0.10057047009468079, 'learning_rate': 0.0001417917314405008, 'epoch': 1.91}
+
 38%|███▊      | 286/750 [30:02<45:53,  5.93s/it]
 38%|███▊      | 287/750 [30:08<45:44,  5.93s/it]
                                                 
{'loss': 0.1937, 'grad_norm': 0.10736412554979324, 'learning_rate': 0.0001415444919648056, 'epoch': 1.91}
+
 38%|███▊      | 287/750 [30:08<45:44,  5.93s/it]
 38%|███▊      | 288/750 [30:14<45:42,  5.94s/it]
                                                 
{'loss': 0.2036, 'grad_norm': 0.10363559424877167, 'learning_rate': 0.00014129854131510975, 'epoch': 1.92}
+
 38%|███▊      | 288/750 [30:14<45:42,  5.94s/it]
 39%|███▊      | 289/750 [30:20<45:36,  5.94s/it]
                                                 
{'loss': 0.1796, 'grad_norm': 0.10259436815977097, 'learning_rate': 0.00014105386833272704, 'epoch': 1.93}
+
 39%|███▊      | 289/750 [30:20<45:36,  5.94s/it]
 39%|███▊      | 290/750 [30:26<45:31,  5.94s/it]
                                                 
{'loss': 0.2107, 'grad_norm': 0.1215776577591896, 'learning_rate': 0.0001408104619937615, 'epoch': 1.93}
+
 39%|███▊      | 290/750 [30:26<45:31,  5.94s/it]
 39%|███▉      | 291/750 [30:31<45:10,  5.91s/it]
                                                 
{'loss': 0.1842, 'grad_norm': 0.11006634682416916, 'learning_rate': 0.00014056831140702128, 'epoch': 1.94}
+
 39%|███▉      | 291/750 [30:31<45:10,  5.91s/it]
 39%|███▉      | 292/750 [30:37<45:05,  5.91s/it]
                                                 
{'loss': 0.2124, 'grad_norm': 0.10849900543689728, 'learning_rate': 0.0001403274058119718, 'epoch': 1.95}
+
 39%|███▉      | 292/750 [30:37<45:05,  5.91s/it]
 39%|███▉      | 293/750 [30:43<45:00,  5.91s/it]
                                                 
{'loss': 0.2102, 'grad_norm': 0.11632896959781647, 'learning_rate': 0.00014008773457672727, 'epoch': 1.95}
+
 39%|███▉      | 293/750 [30:43<45:00,  5.91s/it]
 39%|███▉      | 294/750 [30:49<44:48,  5.90s/it]
                                                 
{'loss': 0.1734, 'grad_norm': 0.10694096237421036, 'learning_rate': 0.00013984928719608012, 'epoch': 1.96}
+
 39%|███▉      | 294/750 [30:49<44:48,  5.90s/it]
 39%|███▉      | 295/750 [30:55<44:48,  5.91s/it]
                                                 
{'loss': 0.2108, 'grad_norm': 0.11361347138881683, 'learning_rate': 0.0001396120532895667, 'epoch': 1.97}
+
 39%|███▉      | 295/750 [30:55<44:48,  5.91s/it]
 39%|███▉      | 296/750 [31:01<44:57,  5.94s/it]
                                                 
{'loss': 0.21, 'grad_norm': 0.1309555172920227, 'learning_rate': 0.00013937602259956938, 'epoch': 1.97}
+
 39%|███▉      | 296/750 [31:01<44:57,  5.94s/it]
 40%|███▉      | 297/750 [31:07<45:19,  6.00s/it]
                                                 
{'loss': 0.1977, 'grad_norm': 0.1072709858417511, 'learning_rate': 0.00013914118498945362, 'epoch': 1.98}
+
 40%|███▉      | 297/750 [31:07<45:19,  6.00s/it]
 40%|███▉      | 298/750 [31:13<45:07,  5.99s/it]
                                                 
{'loss': 0.1671, 'grad_norm': 0.10048194974660873, 'learning_rate': 0.00013890753044173962, 'epoch': 1.99}
+
 40%|███▉      | 298/750 [31:13<45:07,  5.99s/it]
 40%|███▉      | 299/750 [31:19<45:07,  6.00s/it]
                                                 
{'loss': 0.1928, 'grad_norm': 0.10817109793424606, 'learning_rate': 0.0001386750490563073, 'epoch': 1.99}
+
 40%|███▉      | 299/750 [31:19<45:07,  6.00s/it]
 40%|████      | 300/750 [31:27<49:42,  6.63s/it]
                                                 
{'loss': 0.1744, 'grad_norm': 0.10435310751199722, 'learning_rate': 0.00013844373104863458, 'epoch': 2.0}
+
 40%|████      | 300/750 [31:27<49:42,  6.63s/it]
 40%|████      | 301/750 [31:42<1:07:11,  8.98s/it]
                                                   
{'loss': 0.1826, 'grad_norm': 0.10341206192970276, 'learning_rate': 0.00013821356674806781, 'epoch': 2.01}
+
 40%|████      | 301/750 [31:42<1:07:11,  8.98s/it]
 40%|████      | 302/750 [31:48<1:00:09,  8.06s/it]
                                                   
{'loss': 0.1903, 'grad_norm': 0.10279000550508499, 'learning_rate': 0.00013798454659612338, 'epoch': 2.01}
+
 40%|████      | 302/750 [31:48<1:00:09,  8.06s/it]
 40%|████      | 303/750 [31:53<54:51,  7.36s/it]  
                                                 
{'loss': 0.1545, 'grad_norm': 0.10367275774478912, 'learning_rate': 0.00013775666114482078, 'epoch': 2.02}
+
 40%|████      | 303/750 [31:53<54:51,  7.36s/it]
 41%|████      | 304/750 [31:59<51:38,  6.95s/it]
                                                 
{'loss': 0.1745, 'grad_norm': 0.10173984616994858, 'learning_rate': 0.0001375299010550452, 'epoch': 2.03}
+
 41%|████      | 304/750 [31:59<51:38,  6.95s/it]
 41%|████      | 305/750 [32:05<49:18,  6.65s/it]
                                                 
{'loss': 0.173, 'grad_norm': 0.10839492827653885, 'learning_rate': 0.0001373042570949402, 'epoch': 2.03}
+
 41%|████      | 305/750 [32:05<49:18,  6.65s/it]
 41%|████      | 306/750 [32:11<47:39,  6.44s/it]
                                                 
{'loss': 0.1693, 'grad_norm': 0.10868673026561737, 'learning_rate': 0.00013707972013832906, 'epoch': 2.04}
+
 41%|████      | 306/750 [32:11<47:39,  6.44s/it]
 41%|████      | 307/750 [32:17<46:29,  6.30s/it]
                                                 
{'loss': 0.1621, 'grad_norm': 0.11517327278852463, 'learning_rate': 0.0001368562811631643, 'epoch': 2.05}
+
 41%|████      | 307/750 [32:17<46:29,  6.30s/it]
 41%|████      | 308/750 [32:23<45:48,  6.22s/it]
                                                 
{'loss': 0.1666, 'grad_norm': 0.10989931225776672, 'learning_rate': 0.0001366339312500053, 'epoch': 2.05}
+
 41%|████      | 308/750 [32:23<45:48,  6.22s/it]
 41%|████      | 309/750 [32:29<45:01,  6.13s/it]
                                                 
{'loss': 0.1804, 'grad_norm': 0.13048018515110016, 'learning_rate': 0.00013641266158052266, 'epoch': 2.06}
+
 41%|████      | 309/750 [32:29<45:01,  6.13s/it]
 41%|████▏     | 310/750 [32:35<44:25,  6.06s/it]
                                                 
{'loss': 0.1862, 'grad_norm': 0.1161588802933693, 'learning_rate': 0.00013619246343602937, 'epoch': 2.07}
+
 41%|████▏     | 310/750 [32:35<44:25,  6.06s/it]
 41%|████▏     | 311/750 [32:41<44:06,  6.03s/it]
                                                 
{'loss': 0.1887, 'grad_norm': 0.13126613199710846, 'learning_rate': 0.00013597332819603774, 'epoch': 2.07}
+
 41%|████▏     | 311/750 [32:41<44:06,  6.03s/it]
 42%|████▏     | 312/750 [32:47<43:37,  5.98s/it]
                                                 
{'loss': 0.1845, 'grad_norm': 0.12416185438632965, 'learning_rate': 0.00013575524733684213, 'epoch': 2.08}
+
 42%|████▏     | 312/750 [32:47<43:37,  5.98s/it]
 42%|████▏     | 313/750 [32:53<43:16,  5.94s/it]
                                                 
{'loss': 0.1608, 'grad_norm': 0.11586000770330429, 'learning_rate': 0.00013553821243012613, 'epoch': 2.09}
+
 42%|████▏     | 313/750 [32:53<43:16,  5.94s/it]
 42%|████▏     | 314/750 [32:59<43:18,  5.96s/it]
                                                 
{'loss': 0.1742, 'grad_norm': 0.11065978556871414, 'learning_rate': 0.00013532221514159462, 'epoch': 2.09}
+
 42%|████▏     | 314/750 [32:59<43:18,  5.96s/it]
 42%|████▏     | 315/750 [33:05<43:02,  5.94s/it]
                                                 
{'loss': 0.1823, 'grad_norm': 0.12366791069507599, 'learning_rate': 0.00013510724722962957, 'epoch': 2.1}
+
 42%|████▏     | 315/750 [33:05<43:02,  5.94s/it]
 42%|████▏     | 316/750 [33:11<43:06,  5.96s/it]
                                                 
{'loss': 0.1739, 'grad_norm': 0.11502339690923691, 'learning_rate': 0.00013489330054396908, 'epoch': 2.11}
+
 42%|████▏     | 316/750 [33:11<43:06,  5.96s/it]
 42%|████▏     | 317/750 [33:17<43:35,  6.04s/it]
                                                 
{'loss': 0.1851, 'grad_norm': 0.12158779054880142, 'learning_rate': 0.0001346803670244095, 'epoch': 2.11}
+
 42%|████▏     | 317/750 [33:17<43:35,  6.04s/it]
 42%|████▏     | 318/750 [33:23<43:36,  6.06s/it]
                                                 
{'loss': 0.168, 'grad_norm': 0.1147841066122055, 'learning_rate': 0.00013446843869953018, 'epoch': 2.12}
+
 42%|████▏     | 318/750 [33:23<43:36,  6.06s/it]
 43%|████▎     | 319/750 [33:29<43:08,  6.01s/it]
                                                 
{'loss': 0.158, 'grad_norm': 0.11651469022035599, 'learning_rate': 0.0001342575076854399, 'epoch': 2.13}
+
 43%|████▎     | 319/750 [33:29<43:08,  6.01s/it]
 43%|████▎     | 320/750 [33:35<42:49,  5.98s/it]
                                                 
{'loss': 0.1726, 'grad_norm': 0.11769665032625198, 'learning_rate': 0.00013404756618454512, 'epoch': 2.13}
+
 43%|████▎     | 320/750 [33:35<42:49,  5.98s/it]
 43%|████▎     | 321/750 [33:41<42:12,  5.90s/it]
                                                 
{'loss': 0.1777, 'grad_norm': 0.12741614878177643, 'learning_rate': 0.00013383860648433912, 'epoch': 2.14}
+
 43%|████▎     | 321/750 [33:41<42:12,  5.90s/it]
 43%|████▎     | 322/750 [33:47<42:13,  5.92s/it]
                                                 
{'loss': 0.1847, 'grad_norm': 0.12269234657287598, 'learning_rate': 0.0001336306209562122, 'epoch': 2.15}
+
 43%|████▎     | 322/750 [33:47<42:13,  5.92s/it]
 43%|████▎     | 323/750 [33:52<42:06,  5.92s/it]
                                                 
{'loss': 0.1649, 'grad_norm': 0.1309356391429901, 'learning_rate': 0.0001334236020542817, 'epoch': 2.15}
+
 43%|████▎     | 323/750 [33:52<42:06,  5.92s/it]
 43%|████▎     | 324/750 [33:58<42:01,  5.92s/it]
                                                 
{'loss': 0.1771, 'grad_norm': 0.127553790807724, 'learning_rate': 0.00013321754231424221, 'epoch': 2.16}
+
 43%|████▎     | 324/750 [33:58<42:01,  5.92s/it]
 43%|████▎     | 325/750 [34:04<41:54,  5.92s/it]
                                                 
{'loss': 0.1811, 'grad_norm': 0.12544496357440948, 'learning_rate': 0.00013301243435223526, 'epoch': 2.17}
+
 43%|████▎     | 325/750 [34:04<41:54,  5.92s/it]
 43%|████▎     | 326/750 [34:10<41:56,  5.93s/it]
                                                 
{'loss': 0.1734, 'grad_norm': 0.11943470686674118, 'learning_rate': 0.00013280827086373782, 'epoch': 2.17}
+
 43%|████▎     | 326/750 [34:10<41:56,  5.93s/it]
 44%|████▎     | 327/750 [34:16<41:41,  5.91s/it]
                                                 
{'loss': 0.1784, 'grad_norm': 0.12659519910812378, 'learning_rate': 0.00013260504462246983, 'epoch': 2.18}
+
 44%|████▎     | 327/750 [34:16<41:41,  5.91s/it]
 44%|████▎     | 328/750 [34:22<41:48,  5.94s/it]
                                                 
{'loss': 0.1457, 'grad_norm': 0.11461290717124939, 'learning_rate': 0.00013240274847931992, 'epoch': 2.19}
+
 44%|████▎     | 328/750 [34:22<41:48,  5.94s/it]
 44%|████▍     | 329/750 [34:28<41:43,  5.95s/it]
                                                 
{'loss': 0.1749, 'grad_norm': 0.12830543518066406, 'learning_rate': 0.00013220137536128893, 'epoch': 2.19}
+
 44%|████▍     | 329/750 [34:28<41:43,  5.95s/it]
 44%|████▍     | 330/750 [34:34<41:39,  5.95s/it]
                                                 
{'loss': 0.1542, 'grad_norm': 0.12447745352983475, 'learning_rate': 0.00013200091827045155, 'epoch': 2.2}
+
 44%|████▍     | 330/750 [34:34<41:39,  5.95s/it]
 44%|████▍     | 331/750 [34:40<41:32,  5.95s/it]
                                                 
{'loss': 0.1701, 'grad_norm': 0.12327922135591507, 'learning_rate': 0.00013180137028293452, 'epoch': 2.21}
+
 44%|████▍     | 331/750 [34:40<41:32,  5.95s/it]
 44%|████▍     | 332/750 [34:46<41:12,  5.91s/it]
                                                 
{'loss': 0.1665, 'grad_norm': 0.13163968920707703, 'learning_rate': 0.0001316027245479125, 'epoch': 2.21}
+
 44%|████▍     | 332/750 [34:46<41:12,  5.91s/it]
 44%|████▍     | 333/750 [34:52<41:06,  5.92s/it]
                                                 
{'loss': 0.1551, 'grad_norm': 0.11950532346963882, 'learning_rate': 0.00013140497428662003, 'epoch': 2.22}
+
 44%|████▍     | 333/750 [34:52<41:06,  5.92s/it]
 45%|████▍     | 334/750 [34:58<40:45,  5.88s/it]
                                                 
{'loss': 0.1987, 'grad_norm': 0.13608191907405853, 'learning_rate': 0.00013120811279137981, 'epoch': 2.23}
+
 45%|████▍     | 334/750 [34:58<40:45,  5.88s/it]
 45%|████▍     | 335/750 [35:04<41:09,  5.95s/it]
                                                 
{'loss': 0.1631, 'grad_norm': 0.12113139778375626, 'learning_rate': 0.0001310121334246474, 'epoch': 2.23}
+
 45%|████▍     | 335/750 [35:04<41:09,  5.95s/it]
 45%|████▍     | 336/750 [35:10<41:03,  5.95s/it]
                                                 
{'loss': 0.1422, 'grad_norm': 0.11814668029546738, 'learning_rate': 0.0001308170296180709, 'epoch': 2.24}
+
 45%|████▍     | 336/750 [35:10<41:03,  5.95s/it]
 45%|████▍     | 337/750 [35:16<41:13,  5.99s/it]
                                                 
{'loss': 0.1691, 'grad_norm': 0.1233840212225914, 'learning_rate': 0.00013062279487156654, 'epoch': 2.25}
+
 45%|████▍     | 337/750 [35:16<41:13,  5.99s/it]
 45%|████▌     | 338/750 [35:22<40:58,  5.97s/it]
                                                 
{'loss': 0.179, 'grad_norm': 0.13394248485565186, 'learning_rate': 0.00013042942275240903, 'epoch': 2.25}
+
 45%|████▌     | 338/750 [35:22<40:58,  5.97s/it]
 45%|████▌     | 339/750 [35:28<40:47,  5.96s/it]
                                                 
{'loss': 0.1764, 'grad_norm': 0.12861736118793488, 'learning_rate': 0.0001302369068943367, 'epoch': 2.26}
+
 45%|████▌     | 339/750 [35:28<40:47,  5.96s/it]
 45%|████▌     | 340/750 [35:33<40:41,  5.96s/it]
                                                 
{'loss': 0.1584, 'grad_norm': 0.12160958349704742, 'learning_rate': 0.00013004524099667142, 'epoch': 2.27}
+
 45%|████▌     | 340/750 [35:33<40:41,  5.96s/it]
 45%|████▌     | 341/750 [35:39<40:40,  5.97s/it]
                                                 
{'loss': 0.1652, 'grad_norm': 0.12228940427303314, 'learning_rate': 0.00012985441882345223, 'epoch': 2.27}
+
 45%|████▌     | 341/750 [35:39<40:40,  5.97s/it]
 46%|████▌     | 342/750 [35:45<40:35,  5.97s/it]
                                                 
{'loss': 0.1755, 'grad_norm': 0.13051892817020416, 'learning_rate': 0.00012966443420258316, 'epoch': 2.28}
+
 46%|████▌     | 342/750 [35:45<40:35,  5.97s/it]
 46%|████▌     | 343/750 [35:51<40:22,  5.95s/it]
                                                 
{'loss': 0.1669, 'grad_norm': 0.12235318124294281, 'learning_rate': 0.00012947528102499512, 'epoch': 2.29}
+
 46%|████▌     | 343/750 [35:51<40:22,  5.95s/it]
 46%|████▌     | 344/750 [35:57<40:17,  5.96s/it]
                                                 
{'loss': 0.1706, 'grad_norm': 0.12630563974380493, 'learning_rate': 0.00012928695324382025, 'epoch': 2.29}
+
 46%|████▌     | 344/750 [35:57<40:17,  5.96s/it]
 46%|████▌     | 345/750 [36:03<39:59,  5.93s/it]
                                                 
{'loss': 0.1758, 'grad_norm': 0.1292877495288849, 'learning_rate': 0.00012909944487358055, 'epoch': 2.3}
+
 46%|████▌     | 345/750 [36:03<39:59,  5.93s/it]
 46%|████▌     | 346/750 [36:09<39:45,  5.91s/it]
                                                 
{'loss': 0.1732, 'grad_norm': 0.12853533029556274, 'learning_rate': 0.0001289127499893885, 'epoch': 2.31}
+
 46%|████▌     | 346/750 [36:09<39:45,  5.91s/it]
 46%|████▋     | 347/750 [36:15<39:56,  5.95s/it]
                                                 
{'loss': 0.1696, 'grad_norm': 0.12569411098957062, 'learning_rate': 0.00012872686272616092, 'epoch': 2.31}
+
 46%|████▋     | 347/750 [36:15<39:56,  5.95s/it]
 46%|████▋     | 348/750 [36:21<39:45,  5.93s/it]
                                                 
{'loss': 0.1798, 'grad_norm': 0.12562617659568787, 'learning_rate': 0.0001285417772778451, 'epoch': 2.32}
+
 46%|████▋     | 348/750 [36:21<39:45,  5.93s/it]
 47%|████▋     | 349/750 [36:27<39:35,  5.92s/it]
                                                 
{'loss': 0.1729, 'grad_norm': 0.12314952164888382, 'learning_rate': 0.0001283574878966568, 'epoch': 2.33}
+
 47%|████▋     | 349/750 [36:27<39:35,  5.92s/it]
 47%|████▋     | 350/750 [36:33<39:39,  5.95s/it]
                                                 
{'loss': 0.1605, 'grad_norm': 0.11700372397899628, 'learning_rate': 0.00012817398889233115, 'epoch': 2.33}
+
 47%|████▋     | 350/750 [36:33<39:39,  5.95s/it]
 47%|████▋     | 351/750 [36:39<39:40,  5.97s/it]
                                                 
{'loss': 0.1685, 'grad_norm': 0.12326090782880783, 'learning_rate': 0.00012799127463138408, 'epoch': 2.34}
+
 47%|████▋     | 351/750 [36:39<39:40,  5.97s/it]
 47%|████▋     | 352/750 [36:45<39:20,  5.93s/it]
                                                 
{'loss': 0.1725, 'grad_norm': 0.13502374291419983, 'learning_rate': 0.00012780933953638628, 'epoch': 2.35}
+
 47%|████▋     | 352/750 [36:45<39:20,  5.93s/it]
 47%|████▋     | 353/750 [36:51<39:22,  5.95s/it]
                                                 
{'loss': 0.1696, 'grad_norm': 0.12666338682174683, 'learning_rate': 0.00012762817808524798, 'epoch': 2.35}
+
 47%|████▋     | 353/750 [36:51<39:22,  5.95s/it]
 47%|████▋     | 354/750 [36:57<39:38,  6.01s/it]
                                                 
{'loss': 0.1824, 'grad_norm': 0.13442446291446686, 'learning_rate': 0.00012744778481051504, 'epoch': 2.36}
+
 47%|████▋     | 354/750 [36:57<39:38,  6.01s/it]
 47%|████▋     | 355/750 [37:03<39:27,  5.99s/it]
                                                 
{'loss': 0.1508, 'grad_norm': 0.1286042481660843, 'learning_rate': 0.0001272681542986758, 'epoch': 2.37}
+
 47%|████▋     | 355/750 [37:03<39:27,  5.99s/it]
 47%|████▋     | 356/750 [37:09<39:17,  5.98s/it]
                                                 
{'loss': 0.1776, 'grad_norm': 0.13393321633338928, 'learning_rate': 0.0001270892811894789, 'epoch': 2.37}
+
 47%|████▋     | 356/750 [37:09<39:17,  5.98s/it]
 48%|████▊     | 357/750 [37:15<39:08,  5.98s/it]
                                                 
{'loss': 0.1716, 'grad_norm': 0.13838514685630798, 'learning_rate': 0.00012691116017526124, 'epoch': 2.38}
+
 48%|████▊     | 357/750 [37:15<39:08,  5.98s/it]
 48%|████▊     | 358/750 [37:21<39:03,  5.98s/it]
                                                 
{'loss': 0.1916, 'grad_norm': 0.1329745650291443, 'learning_rate': 0.00012673378600028677, 'epoch': 2.39}
+
 48%|████▊     | 358/750 [37:21<39:03,  5.98s/it]
 48%|████▊     | 359/750 [37:27<38:42,  5.94s/it]
                                                 
{'loss': 0.1953, 'grad_norm': 0.13556192815303802, 'learning_rate': 0.00012655715346009488, 'epoch': 2.39}
+
 48%|████▊     | 359/750 [37:27<38:42,  5.94s/it]
 48%|████▊     | 360/750 [37:33<38:37,  5.94s/it]
                                                 
{'loss': 0.1852, 'grad_norm': 0.12661460041999817, 'learning_rate': 0.00012638125740085918, 'epoch': 2.4}
+
 48%|████▊     | 360/750 [37:33<38:37,  5.94s/it]
 48%|████▊     | 361/750 [37:38<38:29,  5.94s/it]
                                                 
{'loss': 0.168, 'grad_norm': 0.13118870556354523, 'learning_rate': 0.0001262060927187558, 'epoch': 2.41}
+
 48%|████▊     | 361/750 [37:38<38:29,  5.94s/it]
 48%|████▊     | 362/750 [37:44<38:25,  5.94s/it]
                                                 
{'loss': 0.1705, 'grad_norm': 0.1364632397890091, 'learning_rate': 0.00012603165435934137, 'epoch': 2.41}
+
 48%|████▊     | 362/750 [37:44<38:25,  5.94s/it]
 48%|████▊     | 363/750 [37:50<38:21,  5.95s/it]
                                                 
{'loss': 0.1681, 'grad_norm': 0.12689675390720367, 'learning_rate': 0.00012585793731694053, 'epoch': 2.42}
+
 48%|████▊     | 363/750 [37:50<38:21,  5.95s/it]
 49%|████▊     | 364/750 [37:56<38:10,  5.93s/it]
                                                 
{'loss': 0.1613, 'grad_norm': 0.1276135891675949, 'learning_rate': 0.00012568493663404258, 'epoch': 2.43}
+
 49%|████▊     | 364/750 [37:56<38:10,  5.93s/it]
 49%|████▊     | 365/750 [38:02<38:07,  5.94s/it]
                                                 
{'loss': 0.1784, 'grad_norm': 0.13087023794651031, 'learning_rate': 0.00012551264740070717, 'epoch': 2.43}
+
 49%|████▊     | 365/750 [38:02<38:07,  5.94s/it]
 49%|████▉     | 366/750 [38:08<38:02,  5.94s/it]
                                                 
{'loss': 0.1973, 'grad_norm': 0.1365162432193756, 'learning_rate': 0.0001253410647539792, 'epoch': 2.44}
+
 49%|████▉     | 366/750 [38:08<38:02,  5.94s/it]
 49%|████▉     | 367/750 [38:14<37:51,  5.93s/it]
                                                 
{'loss': 0.1468, 'grad_norm': 0.12326237559318542, 'learning_rate': 0.00012517018387731205, 'epoch': 2.45}
+
 49%|████▉     | 367/750 [38:14<37:51,  5.93s/it]
 49%|████▉     | 368/750 [38:20<37:46,  5.93s/it]
                                                 
{'loss': 0.1823, 'grad_norm': 0.14036639034748077, 'learning_rate': 0.000125, 'epoch': 2.45}
+
 49%|████▉     | 368/750 [38:20<37:46,  5.93s/it]
 49%|████▉     | 369/750 [38:26<37:44,  5.94s/it]
                                                 
{'loss': 0.1627, 'grad_norm': 0.12352321296930313, 'learning_rate': 0.0001248305083966186, 'epoch': 2.46}
+
 49%|████▉     | 369/750 [38:26<37:44,  5.94s/it]
 49%|████▉     | 370/750 [38:32<37:32,  5.93s/it]
                                                 
{'loss': 0.1674, 'grad_norm': 0.13585619628429413, 'learning_rate': 0.00012466170438647366, 'epoch': 2.47}
+
 49%|████▉     | 370/750 [38:32<37:32,  5.93s/it]
 49%|████▉     | 371/750 [38:38<37:20,  5.91s/it]
                                                 
{'loss': 0.1677, 'grad_norm': 0.12937146425247192, 'learning_rate': 0.00012449358333305828, 'epoch': 2.47}
+
 49%|████▉     | 371/750 [38:38<37:20,  5.91s/it]
 50%|████▉     | 372/750 [38:44<37:12,  5.91s/it]
                                                 
{'loss': 0.1858, 'grad_norm': 0.1331632137298584, 'learning_rate': 0.00012432614064351803, 'epoch': 2.48}
+
 50%|████▉     | 372/750 [38:44<37:12,  5.91s/it]
 50%|████▉     | 373/750 [38:50<37:09,  5.91s/it]
                                                 
{'loss': 0.1632, 'grad_norm': 0.13273197412490845, 'learning_rate': 0.00012415937176812403, 'epoch': 2.49}
+
 50%|████▉     | 373/750 [38:50<37:09,  5.91s/it]
 50%|████▉     | 374/750 [38:56<37:14,  5.94s/it]
                                                 
{'loss': 0.166, 'grad_norm': 0.12437647581100464, 'learning_rate': 0.0001239932721997534, 'epoch': 2.49}
+
 50%|████▉     | 374/750 [38:56<37:14,  5.94s/it]
 50%|█████     | 375/750 [39:02<37:09,  5.94s/it]
                                                 
{'loss': 0.1509, 'grad_norm': 0.12186422199010849, 'learning_rate': 0.00012382783747337805, 'epoch': 2.5}
+
 50%|█████     | 375/750 [39:02<37:09,  5.94s/it]
 50%|█████     | 376/750 [39:08<37:05,  5.95s/it]
                                                 
{'loss': 0.1969, 'grad_norm': 0.1394951343536377, 'learning_rate': 0.0001236630631655603, 'epoch': 2.51}
+
 50%|█████     | 376/750 [39:08<37:05,  5.95s/it]
 50%|█████     | 377/750 [39:13<36:56,  5.94s/it]
                                                 
{'loss': 0.1672, 'grad_norm': 0.1304289847612381, 'learning_rate': 0.0001234989448939562, 'epoch': 2.51}
+
 50%|█████     | 377/750 [39:13<36:56,  5.94s/it]
 50%|█████     | 378/750 [39:19<36:51,  5.94s/it]
                                                 
{'loss': 0.1866, 'grad_norm': 0.13733752071857452, 'learning_rate': 0.00012333547831682581, 'epoch': 2.52}
+
 50%|█████     | 378/750 [39:19<36:51,  5.94s/it]
 51%|█████     | 379/750 [39:25<36:39,  5.93s/it]
                                                 
{'loss': 0.16, 'grad_norm': 0.1302674114704132, 'learning_rate': 0.00012317265913255117, 'epoch': 2.53}
+
 51%|█████     | 379/750 [39:25<36:39,  5.93s/it]
 51%|█████     | 380/750 [39:31<36:29,  5.92s/it]
                                                 
{'loss': 0.1706, 'grad_norm': 0.1256447583436966, 'learning_rate': 0.00012301048307916047, 'epoch': 2.53}
+
 51%|█████     | 380/750 [39:31<36:29,  5.92s/it]
 51%|█████     | 381/750 [39:37<36:29,  5.93s/it]
                                                 
{'loss': 0.1636, 'grad_norm': 0.12781690061092377, 'learning_rate': 0.00012284894593385964, 'epoch': 2.54}
+
 51%|█████     | 381/750 [39:37<36:29,  5.93s/it]
 51%|█████     | 382/750 [39:43<36:13,  5.91s/it]
                                                 
{'loss': 0.1725, 'grad_norm': 0.13997279107570648, 'learning_rate': 0.00012268804351257058, 'epoch': 2.55}
+
 51%|█████     | 382/750 [39:43<36:13,  5.91s/it]
 51%|█████     | 383/750 [39:49<36:13,  5.92s/it]
                                                 
{'loss': 0.1798, 'grad_norm': 0.13648025691509247, 'learning_rate': 0.00012252777166947586, 'epoch': 2.55}
+
 51%|█████     | 383/750 [39:49<36:13,  5.92s/it]
 51%|█████     | 384/750 [39:55<36:10,  5.93s/it]
                                                 
{'loss': 0.1619, 'grad_norm': 0.12897974252700806, 'learning_rate': 0.0001223681262965701, 'epoch': 2.56}
+
 51%|█████     | 384/750 [39:55<36:10,  5.93s/it]
 51%|█████▏    | 385/750 [40:01<36:10,  5.95s/it]
                                                 
{'loss': 0.1682, 'grad_norm': 0.126958429813385, 'learning_rate': 0.00012220910332321784, 'epoch': 2.57}
+
 51%|█████▏    | 385/750 [40:01<36:10,  5.95s/it]
 51%|█████▏    | 386/750 [40:07<35:57,  5.93s/it]
                                                 
{'loss': 0.1846, 'grad_norm': 0.14244182407855988, 'learning_rate': 0.00012205069871571739, 'epoch': 2.57}
+
 51%|█████▏    | 386/750 [40:07<35:57,  5.93s/it]
 52%|█████▏    | 387/750 [40:13<35:52,  5.93s/it]
                                                 
{'loss': 0.1741, 'grad_norm': 0.13084439933300018, 'learning_rate': 0.00012189290847687124, 'epoch': 2.58}
+
 52%|█████▏    | 387/750 [40:13<35:52,  5.93s/it]
 52%|█████▏    | 388/750 [40:19<35:49,  5.94s/it]
                                                 
{'loss': 0.1603, 'grad_norm': 0.12572620809078217, 'learning_rate': 0.0001217357286455623, 'epoch': 2.59}
+
 52%|█████▏    | 388/750 [40:19<35:49,  5.94s/it]
 52%|█████▏    | 389/750 [40:25<35:48,  5.95s/it]
                                                 
{'loss': 0.1558, 'grad_norm': 0.12559637427330017, 'learning_rate': 0.00012157915529633644, 'epoch': 2.59}
+
 52%|█████▏    | 389/750 [40:25<35:48,  5.95s/it]
 52%|█████▏    | 390/750 [40:31<35:54,  5.98s/it]
                                                 
{'loss': 0.1539, 'grad_norm': 0.12262400984764099, 'learning_rate': 0.00012142318453899053, 'epoch': 2.6}
+
 52%|█████▏    | 390/750 [40:31<35:54,  5.98s/it]
 52%|█████▏    | 391/750 [40:37<35:36,  5.95s/it]
                                                 
{'loss': 0.1708, 'grad_norm': 0.13116472959518433, 'learning_rate': 0.00012126781251816648, 'epoch': 2.61}
+
 52%|█████▏    | 391/750 [40:37<35:36,  5.95s/it]
 52%|█████▏    | 392/750 [40:42<35:23,  5.93s/it]
                                                 
{'loss': 0.1699, 'grad_norm': 0.13885435461997986, 'learning_rate': 0.00012111303541295122, 'epoch': 2.61}
+
 52%|█████▏    | 392/750 [40:42<35:23,  5.93s/it]
 52%|█████▏    | 393/750 [40:48<35:25,  5.95s/it]
                                                 
{'loss': 0.1796, 'grad_norm': 0.1409914493560791, 'learning_rate': 0.00012095884943648174, 'epoch': 2.62}
+
 52%|█████▏    | 393/750 [40:48<35:25,  5.95s/it]
 53%|█████▎    | 394/750 [40:54<35:10,  5.93s/it]
                                                 
{'loss': 0.1657, 'grad_norm': 0.12736555933952332, 'learning_rate': 0.0001208052508355561, 'epoch': 2.63}
+
 53%|█████▎    | 394/750 [40:54<35:10,  5.93s/it]
 53%|█████▎    | 395/750 [41:00<35:11,  5.95s/it]
                                                 
{'loss': 0.1533, 'grad_norm': 0.11867203563451767, 'learning_rate': 0.0001206522358902497, 'epoch': 2.63}
+
 53%|█████▎    | 395/750 [41:00<35:11,  5.95s/it]
 53%|█████▎    | 396/750 [41:06<34:55,  5.92s/it]
                                                 
{'loss': 0.145, 'grad_norm': 0.13425129652023315, 'learning_rate': 0.00012049980091353687, 'epoch': 2.64}
+
 53%|█████▎    | 396/750 [41:06<34:55,  5.92s/it]
 53%|█████▎    | 397/750 [41:12<34:43,  5.90s/it]
                                                 
{'loss': 0.1727, 'grad_norm': 0.13228760659694672, 'learning_rate': 0.00012034794225091773, 'epoch': 2.65}
+
 53%|█████▎    | 397/750 [41:12<34:43,  5.90s/it]
 53%|█████▎    | 398/750 [41:18<34:40,  5.91s/it]
                                                 
{'loss': 0.1629, 'grad_norm': 0.12948517501354218, 'learning_rate': 0.00012019665628005017, 'epoch': 2.65}
+
 53%|█████▎    | 398/750 [41:18<34:40,  5.91s/it]
 53%|█████▎    | 399/750 [41:24<34:47,  5.95s/it]
                                                 
{'loss': 0.1785, 'grad_norm': 0.12350346148014069, 'learning_rate': 0.00012004593941038698, 'epoch': 2.66}
+
 53%|█████▎    | 399/750 [41:24<34:47,  5.95s/it]
 53%|█████▎    | 400/750 [41:30<34:42,  5.95s/it]
                                                 
{'loss': 0.167, 'grad_norm': 0.1273069828748703, 'learning_rate': 0.00011989578808281799, 'epoch': 2.67}
+
 53%|█████▎    | 400/750 [41:30<34:42,  5.95s/it][INFO|trainer.py:3831] 2025-06-27 00:30:17,451 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-27 00:30:17,451 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-27 00:30:17,451 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:01<00:07,  1.08it/s]
+
 30%|███       | 3/10 [00:03<00:09,  1.30s/it]
+
 40%|████      | 4/10 [00:05<00:09,  1.50s/it]
+
 50%|█████     | 5/10 [00:07<00:08,  1.62s/it]
+
 60%|██████    | 6/10 [00:09<00:06,  1.70s/it]
+
 70%|███████   | 7/10 [00:11<00:05,  1.74s/it]
+
 80%|████████  | 8/10 [00:12<00:03,  1.77s/it]
+
 90%|█████████ | 9/10 [00:14<00:01,  1.77s/it]
+
100%|██████████| 10/10 [00:18<00:00,  2.35s/it]
                                                 
+
                                               
{'eval_loss': 0.20738115906715393, 'eval_accuracy': 0.22317733268197362, 'eval_runtime': 34.3718, 'eval_samples_per_second': 29.094, 'eval_steps_per_second': 0.291, 'epoch': 2.67}
+
 53%|█████▎    | 400/750 [42:04<34:42,  5.95s/it]
+
100%|██████████| 10/10 [00:24<00:00,  2.35s/it]
+
                                               [INFO|trainer.py:3515] 2025-06-27 00:31:01,624 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-400
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d98ee-0476a86e6a6f1a56406e1487;c8f63932-7692-49dc-a585-c84f146d96b9)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-27 00:31:02,321 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-400/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-27 00:31:02,321 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-400/special_tokens_map.json
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[2025-06-27 00:31:04,807] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Checkpoint global_step400 is begin to save!
+[2025-06-27 00:31:04,832] [INFO] [logging.py:107:log_dist] [Rank 0] Saving model checkpoint: ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d98f1-3de239f907cdb1d94d5c5249;5ba9ed54-ae74-4b07-b188-9efe6bc4b684)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d98f1-4994e1c043686ef74b93d7c3;fd3bf65f-88a1-43b8-892c-7c1cc80492da)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d98f1-7a95ed364dfc844d3064d52f;e3e3af4a-18e4-47d8-abdf-cfe0a5763a38)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d98f1-2861a5886db662a570c575c0;398e435c-532a-4da1-a166-cebd178350ff)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+
 53%|█████▎    | 401/750 [42:24<1:58:37, 20.39s/it]
                                                   
{'loss': 0.1893, 'grad_norm': 0.12802425026893616, 'learning_rate': 0.00011974619876931687, 'epoch': 2.67}
+
 53%|█████▎    | 401/750 [42:24<1:58:37, 20.39s/it]
 54%|█████▎    | 402/750 [42:30<1:33:04, 16.05s/it]
                                                   
{'loss': 0.1748, 'grad_norm': 0.13873571157455444, 'learning_rate': 0.0001195971679725932, 'epoch': 2.68}
+
 54%|█████▎    | 402/750 [42:30<1:33:04, 16.05s/it]
 54%|█████▎    | 403/750 [42:36<1:15:10, 13.00s/it]
                                                   
{'loss': 0.1738, 'grad_norm': 0.13294534385204315, 'learning_rate': 0.00011944869222574892, 'epoch': 2.69}
+
 54%|█████▎    | 403/750 [42:36<1:15:10, 13.00s/it]
 54%|█████▍    | 404/750 [42:42<1:02:47, 10.89s/it]
                                                   
{'loss': 0.1721, 'grad_norm': 0.11917472630739212, 'learning_rate': 0.00011930076809193951, 'epoch': 2.69}
+
 54%|█████▍    | 404/750 [42:42<1:02:47, 10.89s/it]
 54%|█████▍    | 405/750 [42:48<53:55,  9.38s/it]  
                                                 
{'loss': 0.1784, 'grad_norm': 0.13650228083133698, 'learning_rate': 0.0001191533921640401, 'epoch': 2.7}
+
 54%|█████▍    | 405/750 [42:48<53:55,  9.38s/it]
 54%|█████▍    | 406/750 [42:54<47:41,  8.32s/it]
                                                 
{'loss': 0.1786, 'grad_norm': 0.14056342840194702, 'learning_rate': 0.00011900656106431562, 'epoch': 2.71}
+
 54%|█████▍    | 406/750 [42:54<47:41,  8.32s/it]
 54%|█████▍    | 407/750 [42:59<43:29,  7.61s/it]
                                                 
{'loss': 0.161, 'grad_norm': 0.13128921389579773, 'learning_rate': 0.00011886027144409578, 'epoch': 2.71}
+
 54%|█████▍    | 407/750 [42:59<43:29,  7.61s/it]
 54%|█████▍    | 408/750 [43:05<40:41,  7.14s/it]
                                                 
{'loss': 0.1841, 'grad_norm': 0.1360347718000412, 'learning_rate': 0.00011871451998345427, 'epoch': 2.72}
+
 54%|█████▍    | 408/750 [43:06<40:41,  7.14s/it]
 55%|█████▍    | 409/750 [43:11<38:23,  6.76s/it]
                                                 
{'loss': 0.1592, 'grad_norm': 0.13800328969955444, 'learning_rate': 0.00011856930339089229, 'epoch': 2.73}
+
 55%|█████▍    | 409/750 [43:11<38:23,  6.76s/it]
 55%|█████▍    | 410/750 [43:18<37:17,  6.58s/it]
                                                 
{'loss': 0.1935, 'grad_norm': 0.1339549869298935, 'learning_rate': 0.00011842461840302649, 'epoch': 2.73}
+
 55%|█████▍    | 410/750 [43:18<37:17,  6.58s/it]
 55%|█████▍    | 411/750 [43:24<36:21,  6.44s/it]
                                                 
{'loss': 0.1537, 'grad_norm': 0.12085722386837006, 'learning_rate': 0.00011828046178428064, 'epoch': 2.74}
+
 55%|█████▍    | 411/750 [43:24<36:21,  6.44s/it]
 55%|█████▍    | 412/750 [43:30<35:29,  6.30s/it]
                                                 
{'loss': 0.1655, 'grad_norm': 0.1239975243806839, 'learning_rate': 0.00011813683032658212, 'epoch': 2.75}
+
 55%|█████▍    | 412/750 [43:30<35:29,  6.30s/it]
 55%|█████▌    | 413/750 [43:36<34:48,  6.20s/it]
                                                 
{'loss': 0.1696, 'grad_norm': 0.1383960247039795, 'learning_rate': 0.0001179937208490617, 'epoch': 2.75}
+
 55%|█████▌    | 413/750 [43:36<34:48,  6.20s/it]
 55%|█████▌    | 414/750 [43:42<34:20,  6.13s/it]
                                                 
{'loss': 0.1825, 'grad_norm': 0.13461098074913025, 'learning_rate': 0.00011785113019775794, 'epoch': 2.76}
+
 55%|█████▌    | 414/750 [43:42<34:20,  6.13s/it]
 55%|█████▌    | 415/750 [43:48<33:59,  6.09s/it]
                                                 
{'loss': 0.1815, 'grad_norm': 0.13429348170757294, 'learning_rate': 0.00011770905524532507, 'epoch': 2.77}
+
 55%|█████▌    | 415/750 [43:48<33:59,  6.09s/it]
 55%|█████▌    | 416/750 [43:54<33:50,  6.08s/it]
                                                 
{'loss': 0.177, 'grad_norm': 0.13084062933921814, 'learning_rate': 0.00011756749289074503, 'epoch': 2.77}
+
 55%|█████▌    | 416/750 [43:54<33:50,  6.08s/it]
 56%|█████▌    | 417/750 [44:00<33:29,  6.04s/it]
                                                 
{'loss': 0.154, 'grad_norm': 0.1400175243616104, 'learning_rate': 0.00011742644005904313, 'epoch': 2.78}
+
 56%|█████▌    | 417/750 [44:00<33:29,  6.04s/it]
 56%|█████▌    | 418/750 [44:05<33:12,  6.00s/it]
                                                 
{'loss': 0.1561, 'grad_norm': 0.13435116410255432, 'learning_rate': 0.00011728589370100743, 'epoch': 2.79}
+
 56%|█████▌    | 418/750 [44:05<33:12,  6.00s/it]
 56%|█████▌    | 419/750 [44:11<33:02,  5.99s/it]
                                                 
{'loss': 0.1714, 'grad_norm': 0.14085592329502106, 'learning_rate': 0.00011714585079291212, 'epoch': 2.79}
+
 56%|█████▌    | 419/750 [44:11<33:02,  5.99s/it]
 56%|█████▌    | 420/750 [44:17<32:56,  5.99s/it]
                                                 
{'loss': 0.1774, 'grad_norm': 0.13033795356750488, 'learning_rate': 0.00011700630833624395, 'epoch': 2.8}
+
 56%|█████▌    | 420/750 [44:17<32:56,  5.99s/it]
 56%|█████▌    | 421/750 [44:23<32:46,  5.98s/it]
                                                 
{'loss': 0.1757, 'grad_norm': 0.13652119040489197, 'learning_rate': 0.00011686726335743291, 'epoch': 2.81}
+
 56%|█████▌    | 421/750 [44:23<32:46,  5.98s/it]
 56%|█████▋    | 422/750 [44:29<32:50,  6.01s/it]
                                                 
{'loss': 0.185, 'grad_norm': 0.12395608425140381, 'learning_rate': 0.0001167287129075859, 'epoch': 2.81}
+
 56%|█████▋    | 422/750 [44:29<32:50,  6.01s/it]
 56%|█████▋    | 423/750 [44:35<32:41,  6.00s/it]
                                                 
{'loss': 0.1743, 'grad_norm': 0.134079247713089, 'learning_rate': 0.00011659065406222409, 'epoch': 2.82}
+
 56%|█████▋    | 423/750 [44:35<32:41,  6.00s/it]
 57%|█████▋    | 424/750 [44:41<32:32,  5.99s/it]
                                                 
{'loss': 0.1608, 'grad_norm': 0.1343086063861847, 'learning_rate': 0.00011645308392102366, 'epoch': 2.83}
+
 57%|█████▋    | 424/750 [44:41<32:32,  5.99s/it]
 57%|█████▋    | 425/750 [44:47<32:13,  5.95s/it]
                                                 
{'loss': 0.1592, 'grad_norm': 0.12749235332012177, 'learning_rate': 0.00011631599960755992, 'epoch': 2.83}
+
 57%|█████▋    | 425/750 [44:47<32:13,  5.95s/it]
 57%|█████▋    | 426/750 [44:53<32:12,  5.97s/it]
                                                 
{'loss': 0.17, 'grad_norm': 0.12273029237985611, 'learning_rate': 0.00011617939826905469, 'epoch': 2.84}
+
 57%|█████▋    | 426/750 [44:53<32:12,  5.97s/it]
 57%|█████▋    | 427/750 [44:59<32:10,  5.98s/it]
                                                 
{'loss': 0.1658, 'grad_norm': 0.13161002099514008, 'learning_rate': 0.00011604327707612684, 'epoch': 2.85}
+
 57%|█████▋    | 427/750 [44:59<32:10,  5.98s/it]
 57%|█████▋    | 428/750 [45:05<32:20,  6.03s/it]
                                                 
{'loss': 0.1589, 'grad_norm': 0.12151265889406204, 'learning_rate': 0.00011590763322254638, 'epoch': 2.85}
+
 57%|█████▋    | 428/750 [45:05<32:20,  6.03s/it]
 57%|█████▋    | 429/750 [45:11<32:01,  5.99s/it]
                                                 
{'loss': 0.1783, 'grad_norm': 0.14203910529613495, 'learning_rate': 0.00011577246392499127, 'epoch': 2.86}
+
 57%|█████▋    | 429/750 [45:11<32:01,  5.99s/it]
 57%|█████▋    | 430/750 [45:17<32:06,  6.02s/it]
                                                 
{'loss': 0.1545, 'grad_norm': 0.13163644075393677, 'learning_rate': 0.00011563776642280763, 'epoch': 2.87}
+
 57%|█████▋    | 430/750 [45:17<32:06,  6.02s/it]
 57%|█████▋    | 431/750 [45:23<31:56,  6.01s/it]
                                                 
{'loss': 0.1663, 'grad_norm': 0.13118694722652435, 'learning_rate': 0.00011550353797777291, 'epoch': 2.87}
+
 57%|█████▋    | 431/750 [45:23<31:56,  6.01s/it]
 58%|█████▊    | 432/750 [45:29<31:43,  5.99s/it]
                                                 
{'loss': 0.1593, 'grad_norm': 0.12807047367095947, 'learning_rate': 0.00011536977587386216, 'epoch': 2.88}
+
 58%|█████▊    | 432/750 [45:29<31:43,  5.99s/it]
 58%|█████▊    | 433/750 [45:35<31:38,  5.99s/it]
                                                 
{'loss': 0.1518, 'grad_norm': 0.12787529826164246, 'learning_rate': 0.00011523647741701704, 'epoch': 2.89}
+
 58%|█████▊    | 433/750 [45:35<31:38,  5.99s/it]
 58%|█████▊    | 434/750 [45:41<31:31,  5.98s/it]
                                                 
{'loss': 0.178, 'grad_norm': 0.13488884270191193, 'learning_rate': 0.00011510363993491814, 'epoch': 2.89}
+
 58%|█████▊    | 434/750 [45:41<31:31,  5.98s/it]
 58%|█████▊    | 435/750 [45:47<31:25,  5.99s/it]
                                                 
{'loss': 0.178, 'grad_norm': 0.13938601315021515, 'learning_rate': 0.00011497126077675976, 'epoch': 2.9}
+
 58%|█████▊    | 435/750 [45:47<31:25,  5.99s/it]
 58%|█████▊    | 436/750 [45:53<31:11,  5.96s/it]
                                                 
{'loss': 0.1897, 'grad_norm': 0.1396407037973404, 'learning_rate': 0.00011483933731302795, 'epoch': 2.91}
+
 58%|█████▊    | 436/750 [45:53<31:11,  5.96s/it]
 58%|█████▊    | 437/750 [45:59<31:01,  5.95s/it]
                                                 
{'loss': 0.1697, 'grad_norm': 0.12715566158294678, 'learning_rate': 0.00011470786693528087, 'epoch': 2.91}
+
 58%|█████▊    | 437/750 [45:59<31:01,  5.95s/it]
 58%|█████▊    | 438/750 [46:05<30:48,  5.92s/it]
                                                 
{'loss': 0.1879, 'grad_norm': 0.13826040923595428, 'learning_rate': 0.00011457684705593247, 'epoch': 2.92}
+
 58%|█████▊    | 438/750 [46:05<30:48,  5.92s/it]
 59%|█████▊    | 439/750 [46:11<30:44,  5.93s/it]
                                                 
{'loss': 0.1869, 'grad_norm': 0.13141952455043793, 'learning_rate': 0.00011444627510803829, 'epoch': 2.93}
+
 59%|█████▊    | 439/750 [46:11<30:44,  5.93s/it]
 59%|█████▊    | 440/750 [46:17<30:33,  5.91s/it]
                                                 
{'loss': 0.1625, 'grad_norm': 0.1258581280708313, 'learning_rate': 0.00011431614854508447, 'epoch': 2.93}
+
 59%|█████▊    | 440/750 [46:17<30:33,  5.91s/it]
 59%|█████▉    | 441/750 [46:23<30:33,  5.93s/it]
                                                 
{'loss': 0.1499, 'grad_norm': 0.12165071070194244, 'learning_rate': 0.00011418646484077904, 'epoch': 2.94}
+
 59%|█████▉    | 441/750 [46:23<30:33,  5.93s/it]
 59%|█████▉    | 442/750 [46:29<30:30,  5.94s/it]
                                                 
{'loss': 0.1591, 'grad_norm': 0.1242181733250618, 'learning_rate': 0.00011405722148884592, 'epoch': 2.95}
+
 59%|█████▉    | 442/750 [46:29<30:30,  5.94s/it]
 59%|█████▉    | 443/750 [46:35<30:22,  5.94s/it]
                                                 
{'loss': 0.1737, 'grad_norm': 0.13902366161346436, 'learning_rate': 0.00011392841600282156, 'epoch': 2.95}
+
 59%|█████▉    | 443/750 [46:35<30:22,  5.94s/it]
 59%|█████▉    | 444/750 [46:41<30:12,  5.92s/it]
                                                 
{'loss': 0.1972, 'grad_norm': 0.13813108205795288, 'learning_rate': 0.00011380004591585387, 'epoch': 2.96}
+
 59%|█████▉    | 444/750 [46:41<30:12,  5.92s/it]
 59%|█████▉    | 445/750 [46:46<30:05,  5.92s/it]
                                                 
{'loss': 0.1746, 'grad_norm': 0.13776984810829163, 'learning_rate': 0.00011367210878050402, 'epoch': 2.97}
+
 59%|█████▉    | 445/750 [46:46<30:05,  5.92s/it]
 59%|█████▉    | 446/750 [46:52<29:58,  5.91s/it]
                                                 
{'loss': 0.1691, 'grad_norm': 0.13031359016895294, 'learning_rate': 0.00011354460216855028, 'epoch': 2.97}
+
 59%|█████▉    | 446/750 [46:52<29:58,  5.91s/it]
 60%|█████▉    | 447/750 [46:58<30:00,  5.94s/it]
                                                 
{'loss': 0.1791, 'grad_norm': 0.1299438327550888, 'learning_rate': 0.00011341752367079442, 'epoch': 2.98}
+
 60%|█████▉    | 447/750 [46:58<30:00,  5.94s/it]
 60%|█████▉    | 448/750 [47:04<30:10,  6.00s/it]
                                                 
{'loss': 0.175, 'grad_norm': 0.13401517271995544, 'learning_rate': 0.00011329087089687071, 'epoch': 2.99}
+
 60%|█████▉    | 448/750 [47:04<30:10,  6.00s/it]
 60%|█████▉    | 449/750 [47:10<30:07,  6.00s/it]
                                                 
{'loss': 0.1818, 'grad_norm': 0.14243927597999573, 'learning_rate': 0.00011316464147505693, 'epoch': 2.99}
+
 60%|█████▉    | 449/750 [47:10<30:07,  6.00s/it]
 60%|██████    | 450/750 [47:18<32:57,  6.59s/it]
                                                 
{'loss': 0.1652, 'grad_norm': 0.1279725432395935, 'learning_rate': 0.0001130388330520878, 'epoch': 3.0}
+
 60%|██████    | 450/750 [47:18<32:57,  6.59s/it]
 60%|██████    | 451/750 [47:33<44:49,  8.99s/it]
                                                 
{'loss': 0.1655, 'grad_norm': 0.1305849254131317, 'learning_rate': 0.00011291344329297086, 'epoch': 3.01}
+
 60%|██████    | 451/750 [47:33<44:49,  8.99s/it]
 60%|██████    | 452/750 [47:39<40:07,  8.08s/it]
                                                 
{'loss': 0.1494, 'grad_norm': 0.12583260238170624, 'learning_rate': 0.0001127884698808043, 'epoch': 3.01}
+
 60%|██████    | 452/750 [47:39<40:07,  8.08s/it]
 60%|██████    | 453/750 [47:45<36:50,  7.44s/it]
                                                 
{'loss': 0.1321, 'grad_norm': 0.12696702778339386, 'learning_rate': 0.00011266391051659725, 'epoch': 3.02}
+
 60%|██████    | 453/750 [47:45<36:50,  7.44s/it]
 61%|██████    | 454/750 [47:51<34:32,  7.00s/it]
                                                 
{'loss': 0.1292, 'grad_norm': 0.12244350463151932, 'learning_rate': 0.00011253976291909212, 'epoch': 3.03}
+
 61%|██████    | 454/750 [47:51<34:32,  7.00s/it]
 61%|██████    | 455/750 [47:57<32:52,  6.69s/it]
                                                 
{'loss': 0.1409, 'grad_norm': 0.1371058225631714, 'learning_rate': 0.00011241602482458911, 'epoch': 3.03}
+
 61%|██████    | 455/750 [47:57<32:52,  6.69s/it]
 61%|██████    | 456/750 [48:03<31:36,  6.45s/it]
                                                 
{'loss': 0.1306, 'grad_norm': 0.15259167551994324, 'learning_rate': 0.00011229269398677288, 'epoch': 3.04}
+
 61%|██████    | 456/750 [48:03<31:36,  6.45s/it]
 61%|██████    | 457/750 [48:09<30:38,  6.28s/it]
                                                 
{'loss': 0.1329, 'grad_norm': 0.15388865768909454, 'learning_rate': 0.00011216976817654119, 'epoch': 3.05}
+
 61%|██████    | 457/750 [48:09<30:38,  6.28s/it]
 61%|██████    | 458/750 [48:15<30:05,  6.18s/it]
                                                 
{'loss': 0.14, 'grad_norm': 0.15698152780532837, 'learning_rate': 0.00011204724518183582, 'epoch': 3.05}
+
 61%|██████    | 458/750 [48:15<30:05,  6.18s/it]
 61%|██████    | 459/750 [48:21<29:43,  6.13s/it]
                                                 
{'loss': 0.147, 'grad_norm': 0.15454934537410736, 'learning_rate': 0.00011192512280747522, 'epoch': 3.06}
+
 61%|██████    | 459/750 [48:21<29:43,  6.13s/it]
 61%|██████▏   | 460/750 [48:27<29:27,  6.10s/it]
                                                 
{'loss': 0.1612, 'grad_norm': 0.1518414169549942, 'learning_rate': 0.00011180339887498949, 'epoch': 3.07}
+
 61%|██████▏   | 460/750 [48:27<29:27,  6.10s/it]
 61%|██████▏   | 461/750 [48:33<29:06,  6.04s/it]
                                                 
{'loss': 0.1432, 'grad_norm': 0.13643917441368103, 'learning_rate': 0.000111682071222457, 'epoch': 3.07}
+
 61%|██████▏   | 461/750 [48:33<29:06,  6.04s/it]
 62%|██████▏   | 462/750 [48:38<28:46,  6.00s/it]
                                                 
{'loss': 0.1565, 'grad_norm': 0.14681023359298706, 'learning_rate': 0.00011156113770434329, 'epoch': 3.08}
+
 62%|██████▏   | 462/750 [48:38<28:46,  6.00s/it]
 62%|██████▏   | 463/750 [48:44<28:38,  5.99s/it]
                                                 
{'loss': 0.1387, 'grad_norm': 0.13807366788387299, 'learning_rate': 0.00011144059619134151, 'epoch': 3.09}
+
 62%|██████▏   | 463/750 [48:44<28:38,  5.99s/it]
 62%|██████▏   | 464/750 [48:50<28:30,  5.98s/it]
                                                 
{'loss': 0.1257, 'grad_norm': 0.13597892224788666, 'learning_rate': 0.00011132044457021517, 'epoch': 3.09}
+
 62%|██████▏   | 464/750 [48:50<28:30,  5.98s/it]
 62%|██████▏   | 465/750 [48:56<28:13,  5.94s/it]
                                                 
{'loss': 0.1345, 'grad_norm': 0.14555016160011292, 'learning_rate': 0.00011120068074364232, 'epoch': 3.1}
+
 62%|██████▏   | 465/750 [48:56<28:13,  5.94s/it]
 62%|██████▏   | 466/750 [49:02<28:20,  5.99s/it]
                                                 
{'loss': 0.132, 'grad_norm': 0.15187765657901764, 'learning_rate': 0.000111081302630062, 'epoch': 3.11}
+
 62%|██████▏   | 466/750 [49:02<28:20,  5.99s/it]
 62%|██████▏   | 467/750 [49:08<28:12,  5.98s/it]
                                                 
{'loss': 0.1325, 'grad_norm': 0.14446990191936493, 'learning_rate': 0.0001109623081635221, 'epoch': 3.11}
+
 62%|██████▏   | 467/750 [49:08<28:12,  5.98s/it]
 62%|██████▏   | 468/750 [49:14<28:16,  6.02s/it]
                                                 
{'loss': 0.1582, 'grad_norm': 0.16507649421691895, 'learning_rate': 0.00011084369529352937, 'epoch': 3.12}
+
 62%|██████▏   | 468/750 [49:14<28:16,  6.02s/it]
 63%|██████▎   | 469/750 [49:20<27:57,  5.97s/it]
                                                 
{'loss': 0.1382, 'grad_norm': 0.16875247657299042, 'learning_rate': 0.00011072546198490086, 'epoch': 3.13}
+
 63%|██████▎   | 469/750 [49:20<27:57,  5.97s/it]
 63%|██████▎   | 470/750 [49:26<27:48,  5.96s/it]
                                                 
{'loss': 0.1348, 'grad_norm': 0.1724245548248291, 'learning_rate': 0.00011060760621761728, 'epoch': 3.13}
+
 63%|██████▎   | 470/750 [49:26<27:48,  5.96s/it]
 63%|██████▎   | 471/750 [49:32<27:43,  5.96s/it]
                                                 
{'loss': 0.1568, 'grad_norm': 0.15607628226280212, 'learning_rate': 0.00011049012598667817, 'epoch': 3.14}
+
 63%|██████▎   | 471/750 [49:32<27:43,  5.96s/it]
 63%|██████▎   | 472/750 [49:38<27:32,  5.94s/it]
                                                 
{'loss': 0.1417, 'grad_norm': 0.1528671830892563, 'learning_rate': 0.00011037301930195852, 'epoch': 3.15}
+
 63%|██████▎   | 472/750 [49:38<27:32,  5.94s/it]
 63%|██████▎   | 473/750 [49:44<27:17,  5.91s/it]
                                                 
{'loss': 0.1316, 'grad_norm': 0.1656087040901184, 'learning_rate': 0.00011025628418806717, 'epoch': 3.15}
+
 63%|██████▎   | 473/750 [49:44<27:17,  5.91s/it]
 63%|██████▎   | 474/750 [49:50<27:13,  5.92s/it]
                                                 
{'loss': 0.1432, 'grad_norm': 0.15559254586696625, 'learning_rate': 0.00011013991868420692, 'epoch': 3.16}
+
 63%|██████▎   | 474/750 [49:50<27:13,  5.92s/it]
 63%|██████▎   | 475/750 [49:56<27:08,  5.92s/it]
                                                 
{'loss': 0.1329, 'grad_norm': 0.1453288197517395, 'learning_rate': 0.00011002392084403617, 'epoch': 3.17}
+
 63%|██████▎   | 475/750 [49:56<27:08,  5.92s/it]
 63%|██████▎   | 476/750 [50:02<26:56,  5.90s/it]
                                                 
{'loss': 0.1489, 'grad_norm': 0.1557648926973343, 'learning_rate': 0.00010990828873553218, 'epoch': 3.17}
+
 63%|██████▎   | 476/750 [50:02<26:56,  5.90s/it]
 64%|██████▎   | 477/750 [50:07<26:49,  5.90s/it]
                                                 
{'loss': 0.1489, 'grad_norm': 0.15618963539600372, 'learning_rate': 0.00010979302044085591, 'epoch': 3.18}
+
 64%|██████▎   | 477/750 [50:07<26:49,  5.90s/it]
 64%|██████▎   | 478/750 [50:13<26:42,  5.89s/it]
                                                 
{'loss': 0.1365, 'grad_norm': 0.15531198680400848, 'learning_rate': 0.00010967811405621847, 'epoch': 3.19}
+
 64%|██████▎   | 478/750 [50:13<26:42,  5.89s/it]
 64%|██████▍   | 479/750 [50:19<26:43,  5.92s/it]
                                                 
{'loss': 0.1347, 'grad_norm': 0.14251747727394104, 'learning_rate': 0.000109563567691749, 'epoch': 3.19}
+
 64%|██████▍   | 479/750 [50:19<26:43,  5.92s/it]
 64%|██████▍   | 480/750 [50:25<26:43,  5.94s/it]
                                                 
{'loss': 0.1509, 'grad_norm': 0.15274056792259216, 'learning_rate': 0.00010944937947136412, 'epoch': 3.2}
+
 64%|██████▍   | 480/750 [50:25<26:43,  5.94s/it]
 64%|██████▍   | 481/750 [50:31<26:38,  5.94s/it]
                                                 
{'loss': 0.1289, 'grad_norm': 0.14671817421913147, 'learning_rate': 0.00010933554753263895, 'epoch': 3.21}
+
 64%|██████▍   | 481/750 [50:31<26:38,  5.94s/it]
 64%|██████▍   | 482/750 [50:37<26:28,  5.93s/it]
                                                 
{'loss': 0.1385, 'grad_norm': 0.16423137485980988, 'learning_rate': 0.00010922207002667943, 'epoch': 3.21}
+
 64%|██████▍   | 482/750 [50:37<26:28,  5.93s/it]
 64%|██████▍   | 483/750 [50:43<26:20,  5.92s/it]
                                                 
{'loss': 0.135, 'grad_norm': 0.1560370773077011, 'learning_rate': 0.0001091089451179962, 'epoch': 3.22}
+
 64%|██████▍   | 483/750 [50:43<26:20,  5.92s/it]
 65%|██████▍   | 484/750 [50:49<26:16,  5.93s/it]
                                                 
{'loss': 0.1536, 'grad_norm': 0.16102741658687592, 'learning_rate': 0.00010899617098437999, 'epoch': 3.23}
+
 65%|██████▍   | 484/750 [50:49<26:16,  5.93s/it]
 65%|██████▍   | 485/750 [50:55<26:12,  5.93s/it]
                                                 
{'loss': 0.1496, 'grad_norm': 0.1637362390756607, 'learning_rate': 0.00010888374581677828, 'epoch': 3.23}
+
 65%|██████▍   | 485/750 [50:55<26:12,  5.93s/it]
 65%|██████▍   | 486/750 [51:01<26:05,  5.93s/it]
                                                 
{'loss': 0.1405, 'grad_norm': 0.152584508061409, 'learning_rate': 0.00010877166781917343, 'epoch': 3.24}
+
 65%|██████▍   | 486/750 [51:01<26:05,  5.93s/it]
 65%|██████▍   | 487/750 [51:07<26:11,  5.98s/it]
                                                 
{'loss': 0.1574, 'grad_norm': 0.17232313752174377, 'learning_rate': 0.00010865993520846211, 'epoch': 3.25}
+
 65%|██████▍   | 487/750 [51:07<26:11,  5.98s/it]
 65%|██████▌   | 488/750 [51:13<25:58,  5.95s/it]
                                                 
{'loss': 0.142, 'grad_norm': 0.1526651233434677, 'learning_rate': 0.0001085485462143363, 'epoch': 3.25}
+
 65%|██████▌   | 488/750 [51:13<25:58,  5.95s/it]
 65%|██████▌   | 489/750 [51:19<25:47,  5.93s/it]
                                                 
{'loss': 0.1349, 'grad_norm': 0.15860426425933838, 'learning_rate': 0.00010843749907916526, 'epoch': 3.26}
+
 65%|██████▌   | 489/750 [51:19<25:47,  5.93s/it]
 65%|██████▌   | 490/750 [51:25<25:32,  5.89s/it]
                                                 
{'loss': 0.1484, 'grad_norm': 0.16029545664787292, 'learning_rate': 0.0001083267920578793, 'epoch': 3.27}
+
 65%|██████▌   | 490/750 [51:25<25:32,  5.89s/it]
 65%|██████▌   | 491/750 [51:30<25:31,  5.91s/it]
                                                 
{'loss': 0.1388, 'grad_norm': 0.15227065980434418, 'learning_rate': 0.00010821642341785439, 'epoch': 3.27}
+
 65%|██████▌   | 491/750 [51:30<25:31,  5.91s/it]
 66%|██████▌   | 492/750 [51:36<25:26,  5.92s/it]
                                                 
{'loss': 0.156, 'grad_norm': 0.1688096970319748, 'learning_rate': 0.00010810639143879838, 'epoch': 3.28}
+
 66%|██████▌   | 492/750 [51:36<25:26,  5.92s/it]
 66%|██████▌   | 493/750 [51:42<25:17,  5.90s/it]
                                                 
{'loss': 0.1513, 'grad_norm': 0.15941110253334045, 'learning_rate': 0.00010799669441263838, 'epoch': 3.29}
+
 66%|██████▌   | 493/750 [51:42<25:17,  5.90s/it]
 66%|██████▌   | 494/750 [51:48<25:07,  5.89s/it]
                                                 
{'loss': 0.1483, 'grad_norm': 0.16796240210533142, 'learning_rate': 0.0001078873306434093, 'epoch': 3.29}
+
 66%|██████▌   | 494/750 [51:48<25:07,  5.89s/it]
 66%|██████▌   | 495/750 [51:54<25:06,  5.91s/it]
                                                 
{'loss': 0.1561, 'grad_norm': 0.17123353481292725, 'learning_rate': 0.00010777829844714388, 'epoch': 3.3}
+
 66%|██████▌   | 495/750 [51:54<25:06,  5.91s/it]
 66%|██████▌   | 496/750 [52:00<25:06,  5.93s/it]
                                                 
{'loss': 0.134, 'grad_norm': 0.15404053032398224, 'learning_rate': 0.0001076695961517636, 'epoch': 3.31}
+
 66%|██████▌   | 496/750 [52:00<25:06,  5.93s/it]
 66%|██████▋   | 497/750 [52:06<25:02,  5.94s/it]
                                                 
{'loss': 0.1471, 'grad_norm': 0.1669268012046814, 'learning_rate': 0.0001075612220969712, 'epoch': 3.31}
+
 66%|██████▋   | 497/750 [52:06<25:02,  5.94s/it]
 66%|██████▋   | 498/750 [52:12<24:49,  5.91s/it]
                                                 
{'loss': 0.1379, 'grad_norm': 0.1644563376903534, 'learning_rate': 0.00010745317463414389, 'epoch': 3.32}
+
 66%|██████▋   | 498/750 [52:12<24:49,  5.91s/it]
 67%|██████▋   | 499/750 [52:18<24:48,  5.93s/it]
                                                 
{'loss': 0.1357, 'grad_norm': 0.1556372046470642, 'learning_rate': 0.00010734545212622828, 'epoch': 3.33}
+
 67%|██████▋   | 499/750 [52:18<24:48,  5.93s/it]
 67%|██████▋   | 500/750 [52:24<24:37,  5.91s/it]
                                                 
{'loss': 0.1669, 'grad_norm': 0.17315615713596344, 'learning_rate': 0.00010723805294763608, 'epoch': 3.33}
+
 67%|██████▋   | 500/750 [52:24<24:37,  5.91s/it]
 67%|██████▋   | 501/750 [52:30<24:32,  5.91s/it]
                                                 
{'loss': 0.161, 'grad_norm': 0.1610376238822937, 'learning_rate': 0.00010713097548414107, 'epoch': 3.34}
+
 67%|██████▋   | 501/750 [52:30<24:32,  5.91s/it]
 67%|██████▋   | 502/750 [52:36<24:28,  5.92s/it]
                                                 
{'loss': 0.1498, 'grad_norm': 0.16572323441505432, 'learning_rate': 0.00010702421813277718, 'epoch': 3.35}
+
 67%|██████▋   | 502/750 [52:36<24:28,  5.92s/it]
 67%|██████▋   | 503/750 [52:42<24:34,  5.97s/it]
                                                 
{'loss': 0.1544, 'grad_norm': 0.16042861342430115, 'learning_rate': 0.0001069177793017378, 'epoch': 3.35}
+
 67%|██████▋   | 503/750 [52:42<24:34,  5.97s/it]
 67%|██████▋   | 504/750 [52:48<24:26,  5.96s/it]
                                                 
{'loss': 0.1543, 'grad_norm': 0.1622343510389328, 'learning_rate': 0.00010681165741027595, 'epoch': 3.36}
+
 67%|██████▋   | 504/750 [52:48<24:26,  5.96s/it]
 67%|██████▋   | 505/750 [52:54<24:19,  5.96s/it]
                                                 
{'loss': 0.1433, 'grad_norm': 0.15245985984802246, 'learning_rate': 0.00010670585088860586, 'epoch': 3.37}
+
 67%|██████▋   | 505/750 [52:54<24:19,  5.96s/it]
 67%|██████▋   | 506/750 [53:00<24:27,  6.01s/it]
                                                 
{'loss': 0.1526, 'grad_norm': 0.15931057929992676, 'learning_rate': 0.00010660035817780521, 'epoch': 3.37}
+
 67%|██████▋   | 506/750 [53:00<24:27,  6.01s/it]
 68%|██████▊   | 507/750 [53:06<24:19,  6.01s/it]
                                                 
{'loss': 0.1361, 'grad_norm': 0.15014851093292236, 'learning_rate': 0.00010649517772971891, 'epoch': 3.38}
+
 68%|██████▊   | 507/750 [53:06<24:19,  6.01s/it]
 68%|██████▊   | 508/750 [53:12<24:04,  5.97s/it]
                                                 
{'loss': 0.1247, 'grad_norm': 0.16248862445354462, 'learning_rate': 0.00010639030800686347, 'epoch': 3.39}
+
 68%|██████▊   | 508/750 [53:12<24:04,  5.97s/it]
 68%|██████▊   | 509/750 [53:18<23:58,  5.97s/it]
                                                 
{'loss': 0.1353, 'grad_norm': 0.1534966379404068, 'learning_rate': 0.0001062857474823327, 'epoch': 3.39}
+
 68%|██████▊   | 509/750 [53:18<23:58,  5.97s/it]
 68%|██████▊   | 510/750 [53:23<23:51,  5.97s/it]
                                                 
{'loss': 0.1464, 'grad_norm': 0.15869368612766266, 'learning_rate': 0.00010618149463970438, 'epoch': 3.4}
+
 68%|██████▊   | 510/750 [53:24<23:51,  5.97s/it]
 68%|██████▊   | 511/750 [53:29<23:44,  5.96s/it]
                                                 
{'loss': 0.136, 'grad_norm': 0.14489006996154785, 'learning_rate': 0.00010607754797294777, 'epoch': 3.41}
+
 68%|██████▊   | 511/750 [53:29<23:44,  5.96s/it]
 68%|██████▊   | 512/750 [53:35<23:37,  5.95s/it]
                                                 
{'loss': 0.1537, 'grad_norm': 0.15689688920974731, 'learning_rate': 0.00010597390598633231, 'epoch': 3.41}
+
 68%|██████▊   | 512/750 [53:35<23:37,  5.95s/it]
 68%|██████▊   | 513/750 [53:41<23:29,  5.95s/it]
                                                 
{'loss': 0.1497, 'grad_norm': 0.17782750725746155, 'learning_rate': 0.00010587056719433728, 'epoch': 3.42}
+
 68%|██████▊   | 513/750 [53:41<23:29,  5.95s/it]
 69%|██████▊   | 514/750 [53:47<23:21,  5.94s/it]
                                                 
{'loss': 0.1424, 'grad_norm': 0.15875929594039917, 'learning_rate': 0.00010576753012156217, 'epoch': 3.43}
+
 69%|██████▊   | 514/750 [53:47<23:21,  5.94s/it]
 69%|██████▊   | 515/750 [53:53<23:06,  5.90s/it]
                                                 
{'loss': 0.1523, 'grad_norm': 0.1707359254360199, 'learning_rate': 0.0001056647933026385, 'epoch': 3.43}
+
 69%|██████▊   | 515/750 [53:53<23:06,  5.90s/it]
 69%|██████▉   | 516/750 [53:59<22:57,  5.89s/it]
                                                 
{'loss': 0.1594, 'grad_norm': 0.1630873680114746, 'learning_rate': 0.00010556235528214203, 'epoch': 3.44}
+
 69%|██████▉   | 516/750 [53:59<22:57,  5.89s/it]
 69%|██████▉   | 517/750 [54:05<22:56,  5.91s/it]
                                                 
{'loss': 0.1418, 'grad_norm': 0.15407094359397888, 'learning_rate': 0.00010546021461450624, 'epoch': 3.45}
+
 69%|██████▉   | 517/750 [54:05<22:56,  5.91s/it]
 69%|██████▉   | 518/750 [54:11<22:46,  5.89s/it]
                                                 
{'loss': 0.1531, 'grad_norm': 0.1630760133266449, 'learning_rate': 0.00010535836986393678, 'epoch': 3.45}
+
 69%|██████▉   | 518/750 [54:11<22:46,  5.89s/it]
 69%|██████▉   | 519/750 [54:17<22:40,  5.89s/it]
                                                 
{'loss': 0.1638, 'grad_norm': 0.17024356126785278, 'learning_rate': 0.00010525681960432646, 'epoch': 3.46}
+
 69%|██████▉   | 519/750 [54:17<22:40,  5.89s/it]
 69%|██████▉   | 520/750 [54:23<22:38,  5.91s/it]
                                                 
{'loss': 0.1383, 'grad_norm': 0.158006951212883, 'learning_rate': 0.00010515556241917166, 'epoch': 3.47}
+
 69%|██████▉   | 520/750 [54:23<22:38,  5.91s/it]
 69%|██████▉   | 521/750 [54:28<22:30,  5.90s/it]
                                                 
{'loss': 0.1584, 'grad_norm': 0.15718567371368408, 'learning_rate': 0.00010505459690148908, 'epoch': 3.47}
+
 69%|██████▉   | 521/750 [54:28<22:30,  5.90s/it]
 70%|██████▉   | 522/750 [54:34<22:26,  5.91s/it]
                                                 
{'loss': 0.1508, 'grad_norm': 0.16239719092845917, 'learning_rate': 0.00010495392165373376, 'epoch': 3.48}
+
 70%|██████▉   | 522/750 [54:34<22:26,  5.91s/it]
 70%|██████▉   | 523/750 [54:40<22:32,  5.96s/it]
                                                 
{'loss': 0.1432, 'grad_norm': 0.1703788787126541, 'learning_rate': 0.00010485353528771787, 'epoch': 3.49}
+
 70%|██████▉   | 523/750 [54:40<22:32,  5.96s/it]
 70%|██████▉   | 524/750 [54:46<22:22,  5.94s/it]
                                                 
{'loss': 0.1347, 'grad_norm': 0.1606765240430832, 'learning_rate': 0.00010475343642453021, 'epoch': 3.49}
+
 70%|██████▉   | 524/750 [54:46<22:22,  5.94s/it]
 70%|███████   | 525/750 [54:52<22:24,  5.97s/it]
                                                 
{'loss': 0.1377, 'grad_norm': 0.1564657986164093, 'learning_rate': 0.00010465362369445672, 'epoch': 3.5}
+
 70%|███████   | 525/750 [54:52<22:24,  5.97s/it]
 70%|███████   | 526/750 [54:58<22:19,  5.98s/it]
                                                 
{'loss': 0.1491, 'grad_norm': 0.15935762226581573, 'learning_rate': 0.00010455409573690175, 'epoch': 3.51}
+
 70%|███████   | 526/750 [54:58<22:19,  5.98s/it]
 70%|███████   | 527/750 [55:04<22:13,  5.98s/it]
                                                 
{'loss': 0.1531, 'grad_norm': 0.16481848061084747, 'learning_rate': 0.00010445485120031016, 'epoch': 3.51}
+
 70%|███████   | 527/750 [55:04<22:13,  5.98s/it]
 70%|███████   | 528/750 [55:10<22:01,  5.95s/it]
                                                 
{'loss': 0.1392, 'grad_norm': 0.15417121350765228, 'learning_rate': 0.00010435588874209024, 'epoch': 3.52}
+
 70%|███████   | 528/750 [55:10<22:01,  5.95s/it]
 71%|███████   | 529/750 [55:16<21:56,  5.96s/it]
                                                 
{'loss': 0.1455, 'grad_norm': 0.15800175070762634, 'learning_rate': 0.00010425720702853738, 'epoch': 3.53}
+
 71%|███████   | 529/750 [55:16<21:56,  5.96s/it]
 71%|███████   | 530/750 [55:22<21:54,  5.98s/it]
                                                 
{'loss': 0.1533, 'grad_norm': 0.1587381511926651, 'learning_rate': 0.00010415880473475868, 'epoch': 3.53}
+
 71%|███████   | 530/750 [55:22<21:54,  5.98s/it]
 71%|███████   | 531/750 [55:28<21:46,  5.97s/it]
                                                 
{'loss': 0.1462, 'grad_norm': 0.15233056247234344, 'learning_rate': 0.0001040606805445981, 'epoch': 3.54}
+
 71%|███████   | 531/750 [55:28<21:46,  5.97s/it]
 71%|███████   | 532/750 [55:34<21:30,  5.92s/it]
                                                 
{'loss': 0.1415, 'grad_norm': 0.1552983820438385, 'learning_rate': 0.00010396283315056266, 'epoch': 3.55}
+
 71%|███████   | 532/750 [55:34<21:30,  5.92s/it]
 71%|███████   | 533/750 [55:40<21:26,  5.93s/it]
                                                 
{'loss': 0.1435, 'grad_norm': 0.14926421642303467, 'learning_rate': 0.00010386526125374917, 'epoch': 3.55}
+
 71%|███████   | 533/750 [55:40<21:26,  5.93s/it]
 71%|███████   | 534/750 [55:46<21:16,  5.91s/it]
                                                 
{'loss': 0.1498, 'grad_norm': 0.1699218600988388, 'learning_rate': 0.00010376796356377186, 'epoch': 3.56}
+
 71%|███████   | 534/750 [55:46<21:16,  5.91s/it]
 71%|███████▏  | 535/750 [55:52<21:13,  5.92s/it]
                                                 
{'loss': 0.1539, 'grad_norm': 0.16543714702129364, 'learning_rate': 0.00010367093879869068, 'epoch': 3.57}
+
 71%|███████▏  | 535/750 [55:52<21:13,  5.92s/it]
 71%|███████▏  | 536/750 [55:58<21:09,  5.93s/it]
                                                 
{'loss': 0.1517, 'grad_norm': 0.15510369837284088, 'learning_rate': 0.00010357418568494037, 'epoch': 3.57}
+
 71%|███████▏  | 536/750 [55:58<21:09,  5.93s/it]
 72%|███████▏  | 537/750 [56:04<20:57,  5.91s/it]
                                                 
{'loss': 0.1444, 'grad_norm': 0.1694919764995575, 'learning_rate': 0.00010347770295726023, 'epoch': 3.58}
+
 72%|███████▏  | 537/750 [56:04<20:57,  5.91s/it]
 72%|███████▏  | 538/750 [56:09<20:54,  5.92s/it]
                                                 
{'loss': 0.1496, 'grad_norm': 0.1562710702419281, 'learning_rate': 0.00010338148935862473, 'epoch': 3.59}
+
 72%|███████▏  | 538/750 [56:09<20:54,  5.92s/it]
 72%|███████▏  | 539/750 [56:15<20:50,  5.92s/it]
                                                 
{'loss': 0.1501, 'grad_norm': 0.16131797432899475, 'learning_rate': 0.00010328554364017445, 'epoch': 3.59}
+
 72%|███████▏  | 539/750 [56:15<20:50,  5.92s/it]
 72%|███████▏  | 540/750 [56:21<20:47,  5.94s/it]
                                                 
{'loss': 0.1477, 'grad_norm': 0.14995840191841125, 'learning_rate': 0.00010318986456114837, 'epoch': 3.6}
+
 72%|███████▏  | 540/750 [56:21<20:47,  5.94s/it]
 72%|███████▏  | 541/750 [56:28<20:55,  6.01s/it]
                                                 
{'loss': 0.1444, 'grad_norm': 0.15870153903961182, 'learning_rate': 0.00010309445088881625, 'epoch': 3.61}
+
 72%|███████▏  | 541/750 [56:28<20:55,  6.01s/it]
 72%|███████▏  | 542/750 [56:33<20:39,  5.96s/it]
                                                 
{'loss': 0.15, 'grad_norm': 0.16403728723526, 'learning_rate': 0.00010299930139841185, 'epoch': 3.61}
+
 72%|███████▏  | 542/750 [56:33<20:39,  5.96s/it]
 72%|███████▏  | 543/750 [56:39<20:41,  6.00s/it]
                                                 
{'loss': 0.1457, 'grad_norm': 0.14916802942752838, 'learning_rate': 0.00010290441487306716, 'epoch': 3.62}
+
 72%|███████▏  | 543/750 [56:39<20:41,  6.00s/it]
 73%|███████▎  | 544/750 [56:45<20:11,  5.88s/it]
                                                 
{'loss': 0.1304, 'grad_norm': 0.15775884687900543, 'learning_rate': 0.00010280979010374679, 'epoch': 3.63}
+
 73%|███████▎  | 544/750 [56:45<20:11,  5.88s/it]
 73%|███████▎  | 545/750 [56:51<20:03,  5.87s/it]
                                                 
{'loss': 0.1373, 'grad_norm': 0.17496716976165771, 'learning_rate': 0.00010271542588918341, 'epoch': 3.63}
+
 73%|███████▎  | 545/750 [56:51<20:03,  5.87s/it]
 73%|███████▎  | 546/750 [56:57<20:07,  5.92s/it]
                                                 
{'loss': 0.1407, 'grad_norm': 0.15421578288078308, 'learning_rate': 0.00010262132103581367, 'epoch': 3.64}
+
 73%|███████▎  | 546/750 [56:57<20:07,  5.92s/it]
 73%|███████▎  | 547/750 [57:03<20:04,  5.93s/it]
                                                 
{'loss': 0.1456, 'grad_norm': 0.15761473774909973, 'learning_rate': 0.0001025274743577147, 'epoch': 3.65}
+
 73%|███████▎  | 547/750 [57:03<20:04,  5.93s/it]
 73%|███████▎  | 548/750 [57:09<19:58,  5.93s/it]
                                                 
{'loss': 0.1504, 'grad_norm': 0.16617563366889954, 'learning_rate': 0.00010243388467654152, 'epoch': 3.65}
+
 73%|███████▎  | 548/750 [57:09<19:58,  5.93s/it]
 73%|███████▎  | 549/750 [57:15<19:54,  5.95s/it]
                                                 
{'loss': 0.1492, 'grad_norm': 0.1630965769290924, 'learning_rate': 0.0001023405508214647, 'epoch': 3.66}
+
 73%|███████▎  | 549/750 [57:15<19:54,  5.95s/it]
 73%|███████▎  | 550/750 [57:21<19:44,  5.92s/it]
                                                 
{'loss': 0.1787, 'grad_norm': 0.18093156814575195, 'learning_rate': 0.00010224747162910903, 'epoch': 3.67}
+
 73%|███████▎  | 550/750 [57:21<19:44,  5.92s/it]
 73%|███████▎  | 551/750 [57:27<19:38,  5.92s/it]
                                                 
{'loss': 0.1535, 'grad_norm': 0.1675328016281128, 'learning_rate': 0.00010215464594349242, 'epoch': 3.67}
+
 73%|███████▎  | 551/750 [57:27<19:38,  5.92s/it]
 74%|███████▎  | 552/750 [57:33<19:29,  5.91s/it]
                                                 
{'loss': 0.1493, 'grad_norm': 0.1581062376499176, 'learning_rate': 0.00010206207261596577, 'epoch': 3.68}
+
 74%|███████▎  | 552/750 [57:33<19:29,  5.91s/it]
 74%|███████▎  | 553/750 [57:38<19:23,  5.91s/it]
                                                 
{'loss': 0.1624, 'grad_norm': 0.16971106827259064, 'learning_rate': 0.00010196975050515315, 'epoch': 3.69}
+
 74%|███████▎  | 553/750 [57:38<19:23,  5.91s/it]
 74%|███████▍  | 554/750 [57:44<19:20,  5.92s/it]
                                                 
{'loss': 0.147, 'grad_norm': 0.15760289132595062, 'learning_rate': 0.00010187767847689286, 'epoch': 3.69}
+
 74%|███████▍  | 554/750 [57:44<19:20,  5.92s/it]
 74%|███████▍  | 555/750 [57:50<19:11,  5.91s/it]
                                                 
{'loss': 0.1293, 'grad_norm': 0.14435075223445892, 'learning_rate': 0.00010178585540417863, 'epoch': 3.7}
+
 74%|███████▍  | 555/750 [57:50<19:11,  5.91s/it]
 74%|███████▍  | 556/750 [57:56<19:06,  5.91s/it]
                                                 
{'loss': 0.1524, 'grad_norm': 0.1684872806072235, 'learning_rate': 0.00010169428016710201, 'epoch': 3.71}
+
 74%|███████▍  | 556/750 [57:56<19:06,  5.91s/it]
 74%|███████▍  | 557/750 [58:02<18:59,  5.90s/it]
                                                 
{'loss': 0.1433, 'grad_norm': 0.14643168449401855, 'learning_rate': 0.00010160295165279474, 'epoch': 3.71}
+
 74%|███████▍  | 557/750 [58:02<18:59,  5.90s/it]
 74%|███████▍  | 558/750 [58:08<18:50,  5.89s/it]
                                                 
{'loss': 0.1374, 'grad_norm': 0.16298243403434753, 'learning_rate': 0.00010151186875537208, 'epoch': 3.72}
+
 74%|███████▍  | 558/750 [58:08<18:50,  5.89s/it]
 75%|███████▍  | 559/750 [58:14<18:44,  5.89s/it]
                                                 
{'loss': 0.1472, 'grad_norm': 0.1614677906036377, 'learning_rate': 0.00010142103037587655, 'epoch': 3.73}
+
 75%|███████▍  | 559/750 [58:14<18:44,  5.89s/it]
 75%|███████▍  | 560/750 [58:20<18:50,  5.95s/it]
                                                 
{'loss': 0.1409, 'grad_norm': 0.15404939651489258, 'learning_rate': 0.00010133043542222218, 'epoch': 3.73}
+
 75%|███████▍  | 560/750 [58:20<18:50,  5.95s/it]
 75%|███████▍  | 561/750 [58:26<18:45,  5.95s/it]
                                                 
{'loss': 0.1418, 'grad_norm': 0.1949167400598526, 'learning_rate': 0.00010124008280913954, 'epoch': 3.74}
+
 75%|███████▍  | 561/750 [58:26<18:45,  5.95s/it]
 75%|███████▍  | 562/750 [58:32<18:42,  5.97s/it]
                                                 
{'loss': 0.1271, 'grad_norm': 0.15182584524154663, 'learning_rate': 0.00010114997145812079, 'epoch': 3.75}
+
 75%|███████▍  | 562/750 [58:32<18:42,  5.97s/it]
 75%|███████▌  | 563/750 [58:38<18:32,  5.95s/it]
                                                 
{'loss': 0.142, 'grad_norm': 0.17694354057312012, 'learning_rate': 0.000101060100297366, 'epoch': 3.75}
+
 75%|███████▌  | 563/750 [58:38<18:32,  5.95s/it]
 75%|███████▌  | 564/750 [58:44<18:24,  5.94s/it]
                                                 
{'loss': 0.1376, 'grad_norm': 0.16230152547359467, 'learning_rate': 0.00010097046826172943, 'epoch': 3.76}
+
 75%|███████▌  | 564/750 [58:44<18:24,  5.94s/it]
 75%|███████▌  | 565/750 [58:50<18:21,  5.96s/it]
                                                 
{'loss': 0.1398, 'grad_norm': 0.15251514315605164, 'learning_rate': 0.0001008810742926664, 'epoch': 3.77}
+
 75%|███████▌  | 565/750 [58:50<18:21,  5.96s/it]
 75%|███████▌  | 566/750 [58:56<18:12,  5.94s/it]
                                                 
{'loss': 0.1561, 'grad_norm': 0.16820892691612244, 'learning_rate': 0.00010079191733818102, 'epoch': 3.77}
+
 75%|███████▌  | 566/750 [58:56<18:12,  5.94s/it]
 76%|███████▌  | 567/750 [59:02<18:07,  5.94s/it]
                                                 
{'loss': 0.1633, 'grad_norm': 0.1608879268169403, 'learning_rate': 0.00010070299635277398, 'epoch': 3.78}
+
 76%|███████▌  | 567/750 [59:02<18:07,  5.94s/it]
 76%|███████▌  | 568/750 [59:08<18:04,  5.96s/it]
                                                 
{'loss': 0.1519, 'grad_norm': 0.16159476339817047, 'learning_rate': 0.00010061431029739118, 'epoch': 3.79}
+
 76%|███████▌  | 568/750 [59:08<18:04,  5.96s/it]
 76%|███████▌  | 569/750 [59:14<18:02,  5.98s/it]
                                                 
{'loss': 0.1392, 'grad_norm': 0.150252103805542, 'learning_rate': 0.00010052585813937274, 'epoch': 3.79}
+
 76%|███████▌  | 569/750 [59:14<18:02,  5.98s/it]
 76%|███████▌  | 570/750 [59:19<17:52,  5.96s/it]
                                                 
{'loss': 0.1627, 'grad_norm': 0.1768335998058319, 'learning_rate': 0.0001004376388524024, 'epoch': 3.8}
+
 76%|███████▌  | 570/750 [59:19<17:52,  5.96s/it]
 76%|███████▌  | 571/750 [59:25<17:42,  5.94s/it]
                                                 
{'loss': 0.1469, 'grad_norm': 0.15728168189525604, 'learning_rate': 0.00010034965141645762, 'epoch': 3.81}
+
 76%|███████▌  | 571/750 [59:25<17:42,  5.94s/it]
 76%|███████▋  | 572/750 [59:31<17:30,  5.90s/it]
                                                 
{'loss': 0.1472, 'grad_norm': 0.156614288687706, 'learning_rate': 0.00010026189481775992, 'epoch': 3.81}
+
 76%|███████▋  | 572/750 [59:31<17:30,  5.90s/it]
 76%|███████▋  | 573/750 [59:37<17:26,  5.91s/it]
                                                 
{'loss': 0.1467, 'grad_norm': 0.16038380563259125, 'learning_rate': 0.00010017436804872594, 'epoch': 3.82}
+
 76%|███████▋  | 573/750 [59:37<17:26,  5.91s/it]
 77%|███████▋  | 574/750 [59:43<17:16,  5.89s/it]
                                                 
{'loss': 0.1417, 'grad_norm': 0.16885799169540405, 'learning_rate': 0.00010008707010791883, 'epoch': 3.83}
+
 77%|███████▋  | 574/750 [59:43<17:16,  5.89s/it]
 77%|███████▋  | 575/750 [59:49<17:12,  5.90s/it]
                                                 
{'loss': 0.1533, 'grad_norm': 0.15518838167190552, 'learning_rate': 0.0001, 'epoch': 3.83}
+
 77%|███████▋  | 575/750 [59:49<17:12,  5.90s/it]
 77%|███████▋  | 576/750 [59:55<17:03,  5.88s/it]
                                                 
{'loss': 0.1409, 'grad_norm': 0.16575117409229279, 'learning_rate': 9.991315673568166e-05, 'epoch': 3.84}
+
 77%|███████▋  | 576/750 [59:55<17:03,  5.88s/it]
 77%|███████▋  | 577/750 [1:00:01<16:59,  5.89s/it]
                                                   
{'loss': 0.1452, 'grad_norm': 0.16730660200119019, 'learning_rate': 9.982653933167944e-05, 'epoch': 3.85}
+
 77%|███████▋  | 577/750 [1:00:01<16:59,  5.89s/it]
 77%|███████▋  | 578/750 [1:00:07<16:52,  5.89s/it]
                                                   
{'loss': 0.1584, 'grad_norm': 0.1649489849805832, 'learning_rate': 9.97401468106657e-05, 'epoch': 3.85}
+
 77%|███████▋  | 578/750 [1:00:07<16:52,  5.89s/it]
 77%|███████▋  | 579/750 [1:00:13<16:59,  5.96s/it]
                                                   
{'loss': 0.1497, 'grad_norm': 0.16405245661735535, 'learning_rate': 9.965397820122326e-05, 'epoch': 3.86}
+
 77%|███████▋  | 579/750 [1:00:13<16:59,  5.96s/it]
 77%|███████▋  | 580/750 [1:00:19<16:54,  5.97s/it]
                                                   
{'loss': 0.1303, 'grad_norm': 0.15870530903339386, 'learning_rate': 9.956803253779939e-05, 'epoch': 3.87}
+
 77%|███████▋  | 580/750 [1:00:19<16:54,  5.97s/it]
 77%|███████▋  | 581/750 [1:00:25<16:45,  5.95s/it]
                                                   
{'loss': 0.1469, 'grad_norm': 0.16256536543369293, 'learning_rate': 9.94823088606605e-05, 'epoch': 3.87}
+
 77%|███████▋  | 581/750 [1:00:25<16:45,  5.95s/it]
 78%|███████▊  | 582/750 [1:00:30<16:36,  5.93s/it]
                                                   
{'loss': 0.1441, 'grad_norm': 0.1538584679365158, 'learning_rate': 9.939680621584706e-05, 'epoch': 3.88}
+
 78%|███████▊  | 582/750 [1:00:30<16:36,  5.93s/it]
 78%|███████▊  | 583/750 [1:00:36<16:31,  5.94s/it]
                                                   
{'loss': 0.163, 'grad_norm': 0.1761755496263504, 'learning_rate': 9.931152365512906e-05, 'epoch': 3.89}
+
 78%|███████▊  | 583/750 [1:00:36<16:31,  5.94s/it]
 78%|███████▊  | 584/750 [1:00:42<16:26,  5.95s/it]
                                                   
{'loss': 0.1463, 'grad_norm': 0.1586264669895172, 'learning_rate': 9.922646023596178e-05, 'epoch': 3.89}
+
 78%|███████▊  | 584/750 [1:00:42<16:26,  5.95s/it]
 78%|███████▊  | 585/750 [1:00:48<16:18,  5.93s/it]
                                                   
{'loss': 0.1343, 'grad_norm': 0.16324672102928162, 'learning_rate': 9.914161502144209e-05, 'epoch': 3.9}
+
 78%|███████▊  | 585/750 [1:00:48<16:18,  5.93s/it]
 78%|███████▊  | 586/750 [1:00:54<16:11,  5.93s/it]
                                                   
{'loss': 0.1528, 'grad_norm': 0.17567071318626404, 'learning_rate': 9.905698708026503e-05, 'epoch': 3.91}
+
 78%|███████▊  | 586/750 [1:00:54<16:11,  5.93s/it]
 78%|███████▊  | 587/750 [1:01:00<16:12,  5.97s/it]
                                                   
{'loss': 0.1512, 'grad_norm': 0.15445156395435333, 'learning_rate': 9.897257548668106e-05, 'epoch': 3.91}
+
 78%|███████▊  | 587/750 [1:01:00<16:12,  5.97s/it]
 78%|███████▊  | 588/750 [1:01:06<16:02,  5.94s/it]
                                                   
{'loss': 0.1334, 'grad_norm': 0.15348082780838013, 'learning_rate': 9.888837932045328e-05, 'epoch': 3.92}
+
 78%|███████▊  | 588/750 [1:01:06<16:02,  5.94s/it]
 79%|███████▊  | 589/750 [1:01:12<15:56,  5.94s/it]
                                                   
{'loss': 0.1751, 'grad_norm': 0.16937798261642456, 'learning_rate': 9.880439766681545e-05, 'epoch': 3.93}
+
 79%|███████▊  | 589/750 [1:01:12<15:56,  5.94s/it]
 79%|███████▊  | 590/750 [1:01:18<15:52,  5.95s/it]
                                                   
{'loss': 0.1494, 'grad_norm': 0.16821295022964478, 'learning_rate': 9.872062961643026e-05, 'epoch': 3.93}
+
 79%|███████▊  | 590/750 [1:01:18<15:52,  5.95s/it]
 79%|███████▉  | 591/750 [1:01:24<15:41,  5.92s/it]
                                                   
{'loss': 0.1591, 'grad_norm': 0.17528890073299408, 'learning_rate': 9.863707426534786e-05, 'epoch': 3.94}
+
 79%|███████▉  | 591/750 [1:01:24<15:41,  5.92s/it]
 79%|███████▉  | 592/750 [1:01:30<15:32,  5.90s/it]
                                                   
{'loss': 0.1575, 'grad_norm': 0.17449085414409637, 'learning_rate': 9.8553730714965e-05, 'epoch': 3.95}
+
 79%|███████▉  | 592/750 [1:01:30<15:32,  5.90s/it]
 79%|███████▉  | 593/750 [1:01:36<15:27,  5.91s/it]
                                                   
{'loss': 0.1634, 'grad_norm': 0.1672595590353012, 'learning_rate': 9.847059807198442e-05, 'epoch': 3.95}
+
 79%|███████▉  | 593/750 [1:01:36<15:27,  5.91s/it]
 79%|███████▉  | 594/750 [1:01:42<15:25,  5.94s/it]
                                                   
{'loss': 0.1698, 'grad_norm': 0.16503292322158813, 'learning_rate': 9.838767544837452e-05, 'epoch': 3.96}
+
 79%|███████▉  | 594/750 [1:01:42<15:25,  5.94s/it]
 79%|███████▉  | 595/750 [1:01:47<15:14,  5.90s/it]
                                                   
{'loss': 0.1483, 'grad_norm': 0.17282924056053162, 'learning_rate': 9.830496196132975e-05, 'epoch': 3.97}
+
 79%|███████▉  | 595/750 [1:01:47<15:14,  5.90s/it]
 79%|███████▉  | 596/750 [1:01:53<15:13,  5.93s/it]
                                                   
{'loss': 0.1495, 'grad_norm': 0.15530723333358765, 'learning_rate': 9.822245673323086e-05, 'epoch': 3.97}
+
 79%|███████▉  | 596/750 [1:01:53<15:13,  5.93s/it]
 80%|███████▉  | 597/750 [1:02:00<15:15,  5.99s/it]
                                                   
{'loss': 0.1408, 'grad_norm': 0.1515883356332779, 'learning_rate': 9.814015889160605e-05, 'epoch': 3.98}
+
 80%|███████▉  | 597/750 [1:02:00<15:15,  5.99s/it]
 80%|███████▉  | 598/750 [1:02:05<15:05,  5.96s/it]
                                                   
{'loss': 0.1486, 'grad_norm': 0.1659146249294281, 'learning_rate': 9.805806756909202e-05, 'epoch': 3.99}
+
 80%|███████▉  | 598/750 [1:02:05<15:05,  5.96s/it]
 80%|███████▉  | 599/750 [1:02:11<15:00,  5.96s/it]
                                                   
{'loss': 0.1563, 'grad_norm': 0.1595316231250763, 'learning_rate': 9.797618190339569e-05, 'epoch': 3.99}
+
 80%|███████▉  | 599/750 [1:02:11<15:00,  5.96s/it]
 80%|████████  | 600/750 [1:02:20<16:31,  6.61s/it]
                                                   
{'loss': 0.1357, 'grad_norm': 0.1535337269306183, 'learning_rate': 9.789450103725609e-05, 'epoch': 4.0}
+
 80%|████████  | 600/750 [1:02:20<16:31,  6.61s/it][INFO|trainer.py:3831] 2025-06-27 00:51:07,062 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-27 00:51:07,062 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-27 00:51:07,062 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:01<00:07,  1.08it/s]
+
 30%|███       | 3/10 [00:03<00:09,  1.30s/it]
+
 40%|████      | 4/10 [00:05<00:08,  1.50s/it]
+
 50%|█████     | 5/10 [00:07<00:08,  1.62s/it]
+
 60%|██████    | 6/10 [00:09<00:06,  1.69s/it]
+
 70%|███████   | 7/10 [00:11<00:05,  1.74s/it]
+
 80%|████████  | 8/10 [00:12<00:03,  1.77s/it]
+
 90%|█████████ | 9/10 [00:14<00:01,  1.77s/it]
+
100%|██████████| 10/10 [00:18<00:00,  2.32s/it]
                                                   
+
                                               
{'eval_loss': 0.2109662890434265, 'eval_accuracy': 0.22327503663898388, 'eval_runtime': 34.143, 'eval_samples_per_second': 29.289, 'eval_steps_per_second': 0.293, 'epoch': 4.0}
+
 80%|████████  | 600/750 [1:02:54<16:31,  6.61s/it]
+
100%|██████████| 10/10 [00:23<00:00,  2.32s/it]
+
                                               [INFO|trainer.py:3515] 2025-06-27 00:51:50,826 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-600
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d9dce-3f19ec562c8ad0eb1015a041;7fd38569-953b-4fc4-9e3c-f64d3d33a99a)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-27 00:51:51,144 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-600/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-27 00:51:51,144 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-600/special_tokens_map.json
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[2025-06-27 00:51:53,843] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Checkpoint global_step600 is begin to save!
+[2025-06-27 00:51:53,866] [INFO] [logging.py:107:log_dist] [Rank 0] Saving model checkpoint: ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-600/global_step600/mp_rank_00_model_states.pt
+[INFO|trainer.py:3607] 2025-06-27 00:51:53,992 >> Deleting older checkpoint [outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-200] due to args.save_total_limit
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d9dd2-3b36755858a1faf512f6af73;2910ee7c-1f93-404a-a256-b34106f8c3dc)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d9dd2-470bb0bd7346b0fa114344dc;29d37579-b0aa-46a9-b0ec-450107b9a45c)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d9dd2-07f6408654e3f9292492f746;a4377ecd-2002-4607-8a3a-e413c0ff315f)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d9dd2-33813d07324bd09b6b4b232b;03bee699-db56-41b8-9aa2-f89c0b83e37f)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+
 80%|████████  | 601/750 [1:03:21<57:30, 23.16s/it]
                                                   
{'loss': 0.1297, 'grad_norm': 0.16146820783615112, 'learning_rate': 9.781302411840674e-05, 'epoch': 4.01}
+
 80%|████████  | 601/750 [1:03:21<57:30, 23.16s/it]
 80%|████████  | 602/750 [1:03:27<44:21, 17.99s/it]
                                                   
{'loss': 0.1294, 'grad_norm': 0.15809302031993866, 'learning_rate': 9.773175029953825e-05, 'epoch': 4.01}
+
 80%|████████  | 602/750 [1:03:27<44:21, 17.99s/it]
 80%|████████  | 603/750 [1:03:33<35:07, 14.33s/it]
                                                   
{'loss': 0.109, 'grad_norm': 0.15633410215377808, 'learning_rate': 9.76506787382613e-05, 'epoch': 4.02}
+
 80%|████████  | 603/750 [1:03:33<35:07, 14.33s/it]
 81%|████████  | 604/750 [1:03:39<28:45, 11.82s/it]
                                                   
{'loss': 0.1251, 'grad_norm': 0.1713617891073227, 'learning_rate': 9.756980859707e-05, 'epoch': 4.03}
+
 81%|████████  | 604/750 [1:03:39<28:45, 11.82s/it]
 81%|████████  | 605/750 [1:03:45<24:19, 10.06s/it]
                                                   
{'loss': 0.1219, 'grad_norm': 0.19035956263542175, 'learning_rate': 9.748913904330553e-05, 'epoch': 4.03}
+
 81%|████████  | 605/750 [1:03:45<24:19, 10.06s/it]
 81%|████████  | 606/750 [1:03:51<21:13,  8.84s/it]
                                                   
{'loss': 0.1209, 'grad_norm': 0.20892535150051117, 'learning_rate': 9.740866924912017e-05, 'epoch': 4.04}
+
 81%|████████  | 606/750 [1:03:51<21:13,  8.84s/it]
 81%|████████  | 607/750 [1:03:57<19:04,  8.00s/it]
                                                   
{'loss': 0.1172, 'grad_norm': 0.20441478490829468, 'learning_rate': 9.732839839144154e-05, 'epoch': 4.05}
+
 81%|████████  | 607/750 [1:03:57<19:04,  8.00s/it]
 81%|████████  | 608/750 [1:04:03<17:29,  7.39s/it]
                                                   
{'loss': 0.1122, 'grad_norm': 0.20807655155658722, 'learning_rate': 9.724832565193738e-05, 'epoch': 4.05}
+
 81%|████████  | 608/750 [1:04:03<17:29,  7.39s/it]
 81%|████████  | 609/750 [1:04:09<16:22,  6.97s/it]
                                                   
{'loss': 0.1208, 'grad_norm': 0.18891528248786926, 'learning_rate': 9.716845021698033e-05, 'epoch': 4.06}
+
 81%|████████  | 609/750 [1:04:09<16:22,  6.97s/it]
 81%|████████▏ | 610/750 [1:04:15<15:32,  6.66s/it]
                                                   
{'loss': 0.1073, 'grad_norm': 0.17771846055984497, 'learning_rate': 9.708877127761337e-05, 'epoch': 4.07}
+
 81%|████████▏ | 610/750 [1:04:15<15:32,  6.66s/it]
 81%|████████▏ | 611/750 [1:04:21<14:57,  6.46s/it]
                                                   
{'loss': 0.1144, 'grad_norm': 0.16970933973789215, 'learning_rate': 9.700928802951527e-05, 'epoch': 4.07}
+
 81%|████████▏ | 611/750 [1:04:21<14:57,  6.46s/it]
 82%|████████▏ | 612/750 [1:04:27<14:31,  6.31s/it]
                                                   
{'loss': 0.1232, 'grad_norm': 0.18293039500713348, 'learning_rate': 9.69299996729666e-05, 'epoch': 4.08}
+
 82%|████████▏ | 612/750 [1:04:27<14:31,  6.31s/it]
 82%|████████▏ | 613/750 [1:04:33<14:10,  6.21s/it]
                                                   
{'loss': 0.1247, 'grad_norm': 0.172981396317482, 'learning_rate': 9.685090541281594e-05, 'epoch': 4.09}
+
 82%|████████▏ | 613/750 [1:04:33<14:10,  6.21s/it]
 82%|████████▏ | 614/750 [1:04:39<13:56,  6.15s/it]
                                                   
{'loss': 0.1178, 'grad_norm': 0.16729474067687988, 'learning_rate': 9.677200445844624e-05, 'epoch': 4.09}
+
 82%|████████▏ | 614/750 [1:04:39<13:56,  6.15s/it]
 82%|████████▏ | 615/750 [1:04:45<13:43,  6.10s/it]
                                                   
{'loss': 0.1212, 'grad_norm': 0.16877005994319916, 'learning_rate': 9.669329602374179e-05, 'epoch': 4.1}
+
 82%|████████▏ | 615/750 [1:04:45<13:43,  6.10s/it]
 82%|████████▏ | 616/750 [1:04:51<13:42,  6.14s/it]
                                                   
{'loss': 0.1186, 'grad_norm': 0.1810474544763565, 'learning_rate': 9.661477932705526e-05, 'epoch': 4.11}
+
 82%|████████▏ | 616/750 [1:04:51<13:42,  6.14s/it]
 82%|████████▏ | 617/750 [1:04:57<13:31,  6.10s/it]
                                                   
{'loss': 0.138, 'grad_norm': 0.1953781545162201, 'learning_rate': 9.65364535911751e-05, 'epoch': 4.11}
+
 82%|████████▏ | 617/750 [1:04:57<13:31,  6.10s/it]
 82%|████████▏ | 618/750 [1:05:03<13:27,  6.11s/it]
                                                   
{'loss': 0.107, 'grad_norm': 0.1810551881790161, 'learning_rate': 9.645831804329319e-05, 'epoch': 4.12}
+
 82%|████████▏ | 618/750 [1:05:03<13:27,  6.11s/it]
 83%|████████▎ | 619/750 [1:05:09<13:12,  6.05s/it]
                                                   
{'loss': 0.1258, 'grad_norm': 0.21333763003349304, 'learning_rate': 9.63803719149729e-05, 'epoch': 4.13}
+
 83%|████████▎ | 619/750 [1:05:09<13:12,  6.05s/it]
 83%|████████▎ | 620/750 [1:05:15<12:49,  5.92s/it]
                                                   
{'loss': 0.1135, 'grad_norm': 0.1991916447877884, 'learning_rate': 9.630261444211729e-05, 'epoch': 4.13}
+
 83%|████████▎ | 620/750 [1:05:15<12:49,  5.92s/it]
 83%|████████▎ | 621/750 [1:05:21<12:43,  5.92s/it]
                                                   
{'loss': 0.1283, 'grad_norm': 0.19787324965000153, 'learning_rate': 9.622504486493763e-05, 'epoch': 4.14}
+
 83%|████████▎ | 621/750 [1:05:21<12:43,  5.92s/it]
 83%|████████▎ | 622/750 [1:05:27<12:40,  5.95s/it]
                                                   
{'loss': 0.1085, 'grad_norm': 0.17439593374729156, 'learning_rate': 9.614766242792228e-05, 'epoch': 4.15}
+
 83%|████████▎ | 622/750 [1:05:27<12:40,  5.95s/it]
 83%|████████▎ | 623/750 [1:05:33<12:35,  5.95s/it]
                                                   
{'loss': 0.1133, 'grad_norm': 0.1835867315530777, 'learning_rate': 9.607046637980575e-05, 'epoch': 4.15}
+
 83%|████████▎ | 623/750 [1:05:33<12:35,  5.95s/it]
 83%|████████▎ | 624/750 [1:05:39<12:28,  5.94s/it]
                                                   
{'loss': 0.1113, 'grad_norm': 0.1911616176366806, 'learning_rate': 9.599345597353806e-05, 'epoch': 4.16}
+
 83%|████████▎ | 624/750 [1:05:39<12:28,  5.94s/it]
 83%|████████▎ | 625/750 [1:05:45<12:23,  5.95s/it]
                                                   
{'loss': 0.1128, 'grad_norm': 0.19186195731163025, 'learning_rate': 9.59166304662544e-05, 'epoch': 4.17}
+
 83%|████████▎ | 625/750 [1:05:45<12:23,  5.95s/it]
 83%|████████▎ | 626/750 [1:05:50<12:18,  5.95s/it]
                                                   
{'loss': 0.1317, 'grad_norm': 0.20234230160713196, 'learning_rate': 9.5839989119245e-05, 'epoch': 4.17}
+
 83%|████████▎ | 626/750 [1:05:50<12:18,  5.95s/it]
 84%|████████▎ | 627/750 [1:05:56<12:09,  5.93s/it]
                                                   
{'loss': 0.1157, 'grad_norm': 0.1942509412765503, 'learning_rate': 9.57635311979253e-05, 'epoch': 4.18}
+
 84%|████████▎ | 627/750 [1:05:56<12:09,  5.93s/it]
 84%|████████▎ | 628/750 [1:06:02<12:02,  5.92s/it]
                                                   
{'loss': 0.1217, 'grad_norm': 0.1894557774066925, 'learning_rate': 9.568725597180648e-05, 'epoch': 4.19}
+
 84%|████████▎ | 628/750 [1:06:02<12:02,  5.92s/it]
 84%|████████▍ | 629/750 [1:06:08<12:00,  5.96s/it]
                                                   
{'loss': 0.0997, 'grad_norm': 0.16736949980258942, 'learning_rate': 9.561116271446594e-05, 'epoch': 4.19}
+
 84%|████████▍ | 629/750 [1:06:08<12:00,  5.96s/it]
 84%|████████▍ | 630/750 [1:06:14<11:56,  5.97s/it]
                                                   
{'loss': 0.1085, 'grad_norm': 0.1881280392408371, 'learning_rate': 9.553525070351848e-05, 'epoch': 4.2}
+
 84%|████████▍ | 630/750 [1:06:14<11:56,  5.97s/it]
 84%|████████▍ | 631/750 [1:06:20<11:51,  5.98s/it]
                                                   
{'loss': 0.1052, 'grad_norm': 0.19926777482032776, 'learning_rate': 9.54595192205873e-05, 'epoch': 4.21}
+
 84%|████████▍ | 631/750 [1:06:20<11:51,  5.98s/it]
 84%|████████▍ | 632/750 [1:06:26<11:48,  6.00s/it]
                                                   
{'loss': 0.1249, 'grad_norm': 0.19126403331756592, 'learning_rate': 9.538396755127553e-05, 'epoch': 4.21}
+
 84%|████████▍ | 632/750 [1:06:26<11:48,  6.00s/it]
 84%|████████▍ | 633/750 [1:06:32<11:39,  5.98s/it]
                                                   
{'loss': 0.1417, 'grad_norm': 0.2144770473241806, 'learning_rate': 9.530859498513802e-05, 'epoch': 4.22}
+
 84%|████████▍ | 633/750 [1:06:32<11:39,  5.98s/it]
 85%|████████▍ | 634/750 [1:06:38<11:39,  6.03s/it]
                                                   
{'loss': 0.1104, 'grad_norm': 0.20596463978290558, 'learning_rate': 9.523340081565305e-05, 'epoch': 4.23}
+
 85%|████████▍ | 634/750 [1:06:38<11:39,  6.03s/it]
 85%|████████▍ | 635/750 [1:06:44<11:32,  6.02s/it]
                                                   
{'loss': 0.1268, 'grad_norm': 0.19861352443695068, 'learning_rate': 9.515838434019473e-05, 'epoch': 4.23}
+
 85%|████████▍ | 635/750 [1:06:44<11:32,  6.02s/it]
 85%|████████▍ | 636/750 [1:06:50<11:22,  5.99s/it]
                                                   
{'loss': 0.1056, 'grad_norm': 0.1768050193786621, 'learning_rate': 9.508354486000538e-05, 'epoch': 4.24}
+
 85%|████████▍ | 636/750 [1:06:50<11:22,  5.99s/it]
 85%|████████▍ | 637/750 [1:06:56<11:21,  6.03s/it]
                                                   
{'loss': 0.1095, 'grad_norm': 0.17600397765636444, 'learning_rate': 9.500888168016802e-05, 'epoch': 4.25}
+
 85%|████████▍ | 637/750 [1:06:56<11:21,  6.03s/it]
 85%|████████▌ | 638/750 [1:07:02<11:11,  6.00s/it]
                                                   
{'loss': 0.1223, 'grad_norm': 0.1911844164133072, 'learning_rate': 9.493439410957959e-05, 'epoch': 4.25}
+
 85%|████████▌ | 638/750 [1:07:02<11:11,  6.00s/it]
 85%|████████▌ | 639/750 [1:07:08<11:01,  5.96s/it]
                                                   
{'loss': 0.1287, 'grad_norm': 0.20859798789024353, 'learning_rate': 9.48600814609237e-05, 'epoch': 4.26}
+
 85%|████████▌ | 639/750 [1:07:08<11:01,  5.96s/it]
 85%|████████▌ | 640/750 [1:07:14<10:55,  5.96s/it]
                                                   
{'loss': 0.1273, 'grad_norm': 0.19483715295791626, 'learning_rate': 9.478594305064439e-05, 'epoch': 4.27}
+
 85%|████████▌ | 640/750 [1:07:14<10:55,  5.96s/it]
 85%|████████▌ | 641/750 [1:07:20<10:50,  5.97s/it]
                                                   
{'loss': 0.1261, 'grad_norm': 0.19489359855651855, 'learning_rate': 9.471197819891936e-05, 'epoch': 4.27}
+
 85%|████████▌ | 641/750 [1:07:20<10:50,  5.97s/it]
 86%|████████▌ | 642/750 [1:07:26<10:39,  5.92s/it]
                                                   
{'loss': 0.1275, 'grad_norm': 0.21312512457370758, 'learning_rate': 9.463818622963402e-05, 'epoch': 4.28}
+
 86%|████████▌ | 642/750 [1:07:26<10:39,  5.92s/it]
 86%|████████▌ | 643/750 [1:07:32<10:34,  5.93s/it]
                                                   
{'loss': 0.1192, 'grad_norm': 0.21053853631019592, 'learning_rate': 9.45645664703555e-05, 'epoch': 4.29}
+
 86%|████████▌ | 643/750 [1:07:32<10:34,  5.93s/it]
 86%|████████▌ | 644/750 [1:07:38<10:29,  5.94s/it]
                                                   
{'loss': 0.1274, 'grad_norm': 0.19626310467720032, 'learning_rate': 9.44911182523068e-05, 'epoch': 4.29}
+
 86%|████████▌ | 644/750 [1:07:38<10:29,  5.94s/it]
 86%|████████▌ | 645/750 [1:07:44<10:24,  5.94s/it]
                                                   
{'loss': 0.1096, 'grad_norm': 0.19516772031784058, 'learning_rate': 9.441784091034141e-05, 'epoch': 4.3}
+
 86%|████████▌ | 645/750 [1:07:44<10:24,  5.94s/it]
 86%|████████▌ | 646/750 [1:07:50<10:20,  5.97s/it]
                                                   
{'loss': 0.1203, 'grad_norm': 0.1724524199962616, 'learning_rate': 9.434473378291795e-05, 'epoch': 4.31}
+
 86%|████████▌ | 646/750 [1:07:50<10:20,  5.97s/it]
 86%|████████▋ | 647/750 [1:07:56<10:12,  5.95s/it]
                                                   
{'loss': 0.118, 'grad_norm': 0.18564382195472717, 'learning_rate': 9.427179621207505e-05, 'epoch': 4.31}
+
 86%|████████▋ | 647/750 [1:07:56<10:12,  5.95s/it]
 86%|████████▋ | 648/750 [1:08:02<10:08,  5.97s/it]
                                                   
{'loss': 0.1329, 'grad_norm': 0.198990136384964, 'learning_rate': 9.419902754340649e-05, 'epoch': 4.32}
+
 86%|████████▋ | 648/750 [1:08:02<10:08,  5.97s/it]
 87%|████████▋ | 649/750 [1:08:08<10:03,  5.98s/it]
                                                   
{'loss': 0.1229, 'grad_norm': 0.19019605219364166, 'learning_rate': 9.41264271260367e-05, 'epoch': 4.33}
+
 87%|████████▋ | 649/750 [1:08:08<10:03,  5.98s/it]
 87%|████████▋ | 650/750 [1:08:14<09:56,  5.97s/it]
                                                   
{'loss': 0.1068, 'grad_norm': 0.1980881541967392, 'learning_rate': 9.405399431259603e-05, 'epoch': 4.33}
+
 87%|████████▋ | 650/750 [1:08:14<09:56,  5.97s/it]
 87%|████████▋ | 651/750 [1:08:20<09:48,  5.95s/it]
                                                   
{'loss': 0.125, 'grad_norm': 0.19980546832084656, 'learning_rate': 9.398172845919671e-05, 'epoch': 4.34}
+
 87%|████████▋ | 651/750 [1:08:20<09:48,  5.95s/it]
 87%|████████▋ | 652/750 [1:08:26<09:42,  5.94s/it]
                                                   
{'loss': 0.1274, 'grad_norm': 0.20640608668327332, 'learning_rate': 9.390962892540881e-05, 'epoch': 4.35}
+
 87%|████████▋ | 652/750 [1:08:26<09:42,  5.94s/it]
 87%|████████▋ | 653/750 [1:08:31<09:33,  5.91s/it]
                                                   
{'loss': 0.1215, 'grad_norm': 0.20882228016853333, 'learning_rate': 9.383769507423618e-05, 'epoch': 4.35}
+
 87%|████████▋ | 653/750 [1:08:31<09:33,  5.91s/it]
 87%|████████▋ | 654/750 [1:08:37<09:28,  5.92s/it]
                                                   
{'loss': 0.1367, 'grad_norm': 0.2102683037519455, 'learning_rate': 9.376592627209315e-05, 'epoch': 4.36}
+
 87%|████████▋ | 654/750 [1:08:37<09:28,  5.92s/it]
 87%|████████▋ | 655/750 [1:08:43<09:26,  5.96s/it]
                                                   
{'loss': 0.107, 'grad_norm': 0.18556328117847443, 'learning_rate': 9.369432188878082e-05, 'epoch': 4.37}
+
 87%|████████▋ | 655/750 [1:08:43<09:26,  5.96s/it]
 87%|████████▋ | 656/750 [1:08:49<09:16,  5.92s/it]
                                                   
{'loss': 0.1162, 'grad_norm': 0.19150245189666748, 'learning_rate': 9.362288129746395e-05, 'epoch': 4.37}
+
 87%|████████▋ | 656/750 [1:08:49<09:16,  5.92s/it]
 88%|████████▊ | 657/750 [1:08:55<09:10,  5.92s/it]
                                                   
{'loss': 0.1211, 'grad_norm': 0.19354361295700073, 'learning_rate': 9.355160387464784e-05, 'epoch': 4.38}
+
 88%|████████▊ | 657/750 [1:08:55<09:10,  5.92s/it]
 88%|████████▊ | 658/750 [1:09:01<09:05,  5.93s/it]
                                                   
{'loss': 0.1227, 'grad_norm': 0.18872122466564178, 'learning_rate': 9.348048900015556e-05, 'epoch': 4.39}
+
 88%|████████▊ | 658/750 [1:09:01<09:05,  5.93s/it]
 88%|████████▊ | 659/750 [1:09:07<08:59,  5.93s/it]
                                                   
{'loss': 0.1156, 'grad_norm': 0.1734706163406372, 'learning_rate': 9.340953605710524e-05, 'epoch': 4.39}
+
 88%|████████▊ | 659/750 [1:09:07<08:59,  5.93s/it]
 88%|████████▊ | 660/750 [1:09:13<08:52,  5.91s/it]
                                                   
{'loss': 0.1321, 'grad_norm': 0.19426502287387848, 'learning_rate': 9.333874443188752e-05, 'epoch': 4.4}
+
 88%|████████▊ | 660/750 [1:09:13<08:52,  5.91s/it]
 88%|████████▊ | 661/750 [1:09:19<08:48,  5.94s/it]
                                                   
{'loss': 0.1268, 'grad_norm': 0.18161000311374664, 'learning_rate': 9.326811351414343e-05, 'epoch': 4.41}
+
 88%|████████▊ | 661/750 [1:09:19<08:48,  5.94s/it]
 88%|████████▊ | 662/750 [1:09:25<08:44,  5.96s/it]
                                                   
{'loss': 0.1154, 'grad_norm': 0.18544411659240723, 'learning_rate': 9.319764269674211e-05, 'epoch': 4.41}
+
 88%|████████▊ | 662/750 [1:09:25<08:44,  5.96s/it]
 88%|████████▊ | 663/750 [1:09:31<08:34,  5.92s/it]
                                                   
{'loss': 0.1313, 'grad_norm': 0.1945258527994156, 'learning_rate': 9.312733137575907e-05, 'epoch': 4.42}
+
 88%|████████▊ | 663/750 [1:09:31<08:34,  5.92s/it]
 89%|████████▊ | 664/750 [1:09:37<08:30,  5.93s/it]
                                                   
{'loss': 0.1286, 'grad_norm': 0.18810269236564636, 'learning_rate': 9.305717895045427e-05, 'epoch': 4.43}
+
 89%|████████▊ | 664/750 [1:09:37<08:30,  5.93s/it]
 89%|████████▊ | 665/750 [1:09:43<08:25,  5.94s/it]
                                                   
{'loss': 0.1259, 'grad_norm': 0.20028351247310638, 'learning_rate': 9.298718482325068e-05, 'epoch': 4.43}
+
 89%|████████▊ | 665/750 [1:09:43<08:25,  5.94s/it]
 89%|████████▉ | 666/750 [1:09:49<08:18,  5.94s/it]
                                                   
{'loss': 0.1435, 'grad_norm': 0.2324310839176178, 'learning_rate': 9.291734839971293e-05, 'epoch': 4.44}
+
 89%|████████▉ | 666/750 [1:09:49<08:18,  5.94s/it]
 89%|████████▉ | 667/750 [1:09:55<08:13,  5.94s/it]
                                                   
{'loss': 0.1313, 'grad_norm': 0.20299063622951508, 'learning_rate': 9.284766908852593e-05, 'epoch': 4.45}
+
 89%|████████▉ | 667/750 [1:09:55<08:13,  5.94s/it]
 89%|████████▉ | 668/750 [1:10:00<08:05,  5.92s/it]
                                                   
{'loss': 0.1149, 'grad_norm': 0.20010414719581604, 'learning_rate': 9.277814630147407e-05, 'epoch': 4.45}
+
 89%|████████▉ | 668/750 [1:10:00<08:05,  5.92s/it]
 89%|████████▉ | 669/750 [1:10:06<07:58,  5.91s/it]
                                                   
{'loss': 0.127, 'grad_norm': 0.20554889738559723, 'learning_rate': 9.270877945342015e-05, 'epoch': 4.46}
+
 89%|████████▉ | 669/750 [1:10:06<07:58,  5.91s/it]
 89%|████████▉ | 670/750 [1:10:12<07:53,  5.91s/it]
                                                   
{'loss': 0.1151, 'grad_norm': 0.19019319117069244, 'learning_rate': 9.263956796228493e-05, 'epoch': 4.47}
+
 89%|████████▉ | 670/750 [1:10:12<07:53,  5.91s/it]
 89%|████████▉ | 671/750 [1:10:18<07:45,  5.90s/it]
                                                   
{'loss': 0.1133, 'grad_norm': 0.1894761323928833, 'learning_rate': 9.257051124902634e-05, 'epoch': 4.47}
+
 89%|████████▉ | 671/750 [1:10:18<07:45,  5.90s/it]
 90%|████████▉ | 672/750 [1:10:24<07:41,  5.92s/it]
                                                   
{'loss': 0.1227, 'grad_norm': 0.21622046828269958, 'learning_rate': 9.250160873761937e-05, 'epoch': 4.48}
+
 90%|████████▉ | 672/750 [1:10:24<07:41,  5.92s/it]
 90%|████████▉ | 673/750 [1:10:30<07:37,  5.94s/it]
                                                   
{'loss': 0.1096, 'grad_norm': 0.18021948635578156, 'learning_rate': 9.243285985503578e-05, 'epoch': 4.49}
+
 90%|████████▉ | 673/750 [1:10:30<07:37,  5.94s/it]
 90%|████████▉ | 674/750 [1:10:36<07:33,  5.97s/it]
                                                   
{'loss': 0.1162, 'grad_norm': 0.18876951932907104, 'learning_rate': 9.236426403122409e-05, 'epoch': 4.49}
+
 90%|████████▉ | 674/750 [1:10:36<07:33,  5.97s/it]
 90%|█████████ | 675/750 [1:10:42<07:25,  5.94s/it]
                                                   
{'loss': 0.1111, 'grad_norm': 0.19144152104854584, 'learning_rate': 9.229582069908974e-05, 'epoch': 4.5}
+
 90%|█████████ | 675/750 [1:10:42<07:25,  5.94s/it]
 90%|█████████ | 676/750 [1:10:48<07:19,  5.95s/it]
                                                   
{'loss': 0.1304, 'grad_norm': 0.20279021561145782, 'learning_rate': 9.222752929447538e-05, 'epoch': 4.51}
+
 90%|█████████ | 676/750 [1:10:48<07:19,  5.95s/it]
 90%|█████████ | 677/750 [1:10:54<07:15,  5.96s/it]
                                                   
{'loss': 0.11, 'grad_norm': 0.19248034060001373, 'learning_rate': 9.215938925614141e-05, 'epoch': 4.51}
+
 90%|█████████ | 677/750 [1:10:54<07:15,  5.96s/it]
 90%|█████████ | 678/750 [1:11:00<07:09,  5.96s/it]
                                                   
{'loss': 0.1148, 'grad_norm': 0.19480571150779724, 'learning_rate': 9.209140002574652e-05, 'epoch': 4.52}
+
 90%|█████████ | 678/750 [1:11:00<07:09,  5.96s/it]
 91%|█████████ | 679/750 [1:11:06<07:03,  5.96s/it]
                                                   
{'loss': 0.1315, 'grad_norm': 0.2050219625234604, 'learning_rate': 9.20235610478285e-05, 'epoch': 4.53}
+
 91%|█████████ | 679/750 [1:11:06<07:03,  5.96s/it]
 91%|█████████ | 680/750 [1:11:12<06:56,  5.95s/it]
                                                   
{'loss': 0.1219, 'grad_norm': 0.1935853213071823, 'learning_rate': 9.19558717697852e-05, 'epoch': 4.53}
+
 91%|█████████ | 680/750 [1:11:12<06:56,  5.95s/it]
 91%|█████████ | 681/750 [1:11:18<06:48,  5.92s/it]
                                                   
{'loss': 0.1207, 'grad_norm': 0.2061282992362976, 'learning_rate': 9.18883316418556e-05, 'epoch': 4.54}
+
 91%|█████████ | 681/750 [1:11:18<06:48,  5.92s/it]
 91%|█████████ | 682/750 [1:11:24<06:43,  5.93s/it]
                                                   
{'loss': 0.1143, 'grad_norm': 0.19050821661949158, 'learning_rate': 9.182094011710111e-05, 'epoch': 4.55}
+
 91%|█████████ | 682/750 [1:11:24<06:43,  5.93s/it]
 91%|█████████ | 683/750 [1:11:29<06:36,  5.92s/it]
                                                   
{'loss': 0.124, 'grad_norm': 0.1824985146522522, 'learning_rate': 9.175369665138691e-05, 'epoch': 4.55}
+
 91%|█████████ | 683/750 [1:11:29<06:36,  5.92s/it]
 91%|█████████ | 684/750 [1:11:35<06:31,  5.93s/it]
                                                   
{'loss': 0.1162, 'grad_norm': 0.18643681704998016, 'learning_rate': 9.168660070336348e-05, 'epoch': 4.56}
+
 91%|█████████ | 684/750 [1:11:35<06:31,  5.93s/it]
 91%|█████████▏| 685/750 [1:11:41<06:23,  5.91s/it]
                                                   
{'loss': 0.1196, 'grad_norm': 0.2066197693347931, 'learning_rate': 9.161965173444834e-05, 'epoch': 4.57}
+
 91%|█████████▏| 685/750 [1:11:41<06:23,  5.91s/it]
 91%|█████████▏| 686/750 [1:11:47<06:18,  5.91s/it]
                                                   
{'loss': 0.1396, 'grad_norm': 0.23195287585258484, 'learning_rate': 9.155284920880796e-05, 'epoch': 4.57}
+
 91%|█████████▏| 686/750 [1:11:47<06:18,  5.91s/it]
 92%|█████████▏| 687/750 [1:11:53<06:13,  5.93s/it]
                                                   
{'loss': 0.1106, 'grad_norm': 0.18247000873088837, 'learning_rate': 9.148619259333955e-05, 'epoch': 4.58}
+
 92%|█████████▏| 687/750 [1:11:53<06:13,  5.93s/it]
 92%|█████████▏| 688/750 [1:11:59<06:08,  5.94s/it]
                                                   
{'loss': 0.1161, 'grad_norm': 0.1833108812570572, 'learning_rate': 9.141968135765342e-05, 'epoch': 4.59}
+
 92%|█████████▏| 688/750 [1:11:59<06:08,  5.94s/it]
 92%|█████████▏| 689/750 [1:12:05<06:00,  5.91s/it]
                                                   
{'loss': 0.116, 'grad_norm': 0.19194342195987701, 'learning_rate': 9.135331497405505e-05, 'epoch': 4.59}
+
 92%|█████████▏| 689/750 [1:12:05<06:00,  5.91s/it]
 92%|█████████▏| 690/750 [1:12:11<05:54,  5.90s/it]
                                                   
{'loss': 0.1349, 'grad_norm': 0.20306549966335297, 'learning_rate': 9.128709291752768e-05, 'epoch': 4.6}
+
 92%|█████████▏| 690/750 [1:12:11<05:54,  5.90s/it]
 92%|█████████▏| 691/750 [1:12:17<05:49,  5.92s/it]
                                                   
{'loss': 0.1342, 'grad_norm': 0.19383618235588074, 'learning_rate': 9.122101466571477e-05, 'epoch': 4.61}
+
 92%|█████████▏| 691/750 [1:12:17<05:49,  5.92s/it]
 92%|█████████▏| 692/750 [1:12:23<05:45,  5.96s/it]
                                                   
{'loss': 0.1214, 'grad_norm': 0.18854767084121704, 'learning_rate': 9.115507969890265e-05, 'epoch': 4.61}
+
 92%|█████████▏| 692/750 [1:12:23<05:45,  5.96s/it]
 92%|█████████▏| 693/750 [1:12:29<05:38,  5.95s/it]
                                                   
{'loss': 0.133, 'grad_norm': 0.2005634307861328, 'learning_rate': 9.108928750000352e-05, 'epoch': 4.62}
+
 92%|█████████▏| 693/750 [1:12:29<05:38,  5.95s/it]
 93%|█████████▎| 694/750 [1:12:35<05:31,  5.92s/it]
                                                   
{'loss': 0.1229, 'grad_norm': 0.19461499154567719, 'learning_rate': 9.102363755453824e-05, 'epoch': 4.63}
+
 93%|█████████▎| 694/750 [1:12:35<05:31,  5.92s/it]
 93%|█████████▎| 695/750 [1:12:41<05:26,  5.93s/it]
                                                   
{'loss': 0.1194, 'grad_norm': 0.1921541690826416, 'learning_rate': 9.095812935061951e-05, 'epoch': 4.63}
+
 93%|█████████▎| 695/750 [1:12:41<05:26,  5.93s/it]
 93%|█████████▎| 696/750 [1:12:47<05:20,  5.93s/it]
                                                   
{'loss': 0.1241, 'grad_norm': 0.1985134333372116, 'learning_rate': 9.089276237893512e-05, 'epoch': 4.64}
+
 93%|█████████▎| 696/750 [1:12:47<05:20,  5.93s/it]
 93%|█████████▎| 697/750 [1:12:52<05:13,  5.91s/it]
                                                   
{'loss': 0.1144, 'grad_norm': 0.20523406565189362, 'learning_rate': 9.082753613273131e-05, 'epoch': 4.65}
+
 93%|█████████▎| 697/750 [1:12:52<05:13,  5.91s/it]
 93%|█████████▎| 698/750 [1:12:58<05:06,  5.89s/it]
                                                   
{'loss': 0.1206, 'grad_norm': 0.19906945526599884, 'learning_rate': 9.076245010779625e-05, 'epoch': 4.65}
+
 93%|█████████▎| 698/750 [1:12:58<05:06,  5.89s/it]
 93%|█████████▎| 699/750 [1:13:04<05:01,  5.91s/it]
                                                   
{'loss': 0.1092, 'grad_norm': 0.198943093419075, 'learning_rate': 9.069750380244364e-05, 'epoch': 4.66}
+
 93%|█████████▎| 699/750 [1:13:04<05:01,  5.91s/it]
 93%|█████████▎| 700/750 [1:13:10<04:56,  5.93s/it]
                                                   
{'loss': 0.1142, 'grad_norm': 0.19928747415542603, 'learning_rate': 9.063269671749657e-05, 'epoch': 4.67}
+
 93%|█████████▎| 700/750 [1:13:10<04:56,  5.93s/it]
 93%|█████████▎| 701/750 [1:13:16<04:49,  5.92s/it]
                                                   
{'loss': 0.1122, 'grad_norm': 0.17332346737384796, 'learning_rate': 9.056802835627134e-05, 'epoch': 4.67}
+
 93%|█████████▎| 701/750 [1:13:16<04:49,  5.92s/it]
 94%|█████████▎| 702/750 [1:13:22<04:45,  5.94s/it]
                                                   
{'loss': 0.1274, 'grad_norm': 0.2081272155046463, 'learning_rate': 9.050349822456143e-05, 'epoch': 4.68}
+
 94%|█████████▎| 702/750 [1:13:22<04:45,  5.94s/it]
 94%|█████████▎| 703/750 [1:13:28<04:38,  5.92s/it]
                                                   
{'loss': 0.1178, 'grad_norm': 0.21539074182510376, 'learning_rate': 9.043910583062168e-05, 'epoch': 4.69}
+
 94%|█████████▎| 703/750 [1:13:28<04:38,  5.92s/it]
 94%|█████████▍| 704/750 [1:13:34<04:31,  5.91s/it]
                                                   
{'loss': 0.1394, 'grad_norm': 0.21388739347457886, 'learning_rate': 9.037485068515265e-05, 'epoch': 4.69}
+
 94%|█████████▍| 704/750 [1:13:34<04:31,  5.91s/it]
 94%|█████████▍| 705/750 [1:13:40<04:24,  5.88s/it]
                                                   
{'loss': 0.129, 'grad_norm': 0.2084227055311203, 'learning_rate': 9.03107323012848e-05, 'epoch': 4.7}
+
 94%|█████████▍| 705/750 [1:13:40<04:24,  5.88s/it]
 94%|█████████▍| 706/750 [1:13:46<04:20,  5.92s/it]
                                                   
{'loss': 0.1305, 'grad_norm': 0.18557515740394592, 'learning_rate': 9.024675019456317e-05, 'epoch': 4.71}
+
 94%|█████████▍| 706/750 [1:13:46<04:20,  5.92s/it]
 94%|█████████▍| 707/750 [1:13:52<04:13,  5.91s/it]
                                                   
{'loss': 0.1167, 'grad_norm': 0.19854147732257843, 'learning_rate': 9.01829038829319e-05, 'epoch': 4.71}
+
 94%|█████████▍| 707/750 [1:13:52<04:13,  5.91s/it]
 94%|█████████▍| 708/750 [1:13:57<04:07,  5.90s/it]
                                                   
{'loss': 0.1169, 'grad_norm': 0.18760545551776886, 'learning_rate': 9.011919288671903e-05, 'epoch': 4.72}
+
 94%|█████████▍| 708/750 [1:13:57<04:07,  5.90s/it]
 95%|█████████▍| 709/750 [1:14:03<04:01,  5.90s/it]
                                                   
{'loss': 0.1275, 'grad_norm': 0.17979490756988525, 'learning_rate': 9.005561672862141e-05, 'epoch': 4.73}
+
 95%|█████████▍| 709/750 [1:14:03<04:01,  5.90s/it]
 95%|█████████▍| 710/750 [1:14:09<03:56,  5.92s/it]
                                                   
{'loss': 0.1218, 'grad_norm': 0.1948082447052002, 'learning_rate': 8.999217493368951e-05, 'epoch': 4.73}
+
 95%|█████████▍| 710/750 [1:14:09<03:56,  5.92s/it]
 95%|█████████▍| 711/750 [1:14:15<03:51,  5.95s/it]
                                                   
{'loss': 0.133, 'grad_norm': 0.20126453042030334, 'learning_rate': 8.992886702931274e-05, 'epoch': 4.74}
+
 95%|█████████▍| 711/750 [1:14:15<03:51,  5.95s/it]
 95%|█████████▍| 712/750 [1:14:21<03:46,  5.97s/it]
                                                   
{'loss': 0.1281, 'grad_norm': 0.19406697154045105, 'learning_rate': 8.986569254520445e-05, 'epoch': 4.75}
+
 95%|█████████▍| 712/750 [1:14:21<03:46,  5.97s/it]
 95%|█████████▌| 713/750 [1:14:27<03:41,  5.97s/it]
                                                   
{'loss': 0.1313, 'grad_norm': 0.1871439665555954, 'learning_rate': 8.980265101338746e-05, 'epoch': 4.75}
+
 95%|█████████▌| 713/750 [1:14:27<03:41,  5.97s/it]
 95%|█████████▌| 714/750 [1:14:33<03:34,  5.96s/it]
                                                   
{'loss': 0.1083, 'grad_norm': 0.17861828207969666, 'learning_rate': 8.973974196817932e-05, 'epoch': 4.76}
+
 95%|█████████▌| 714/750 [1:14:33<03:34,  5.96s/it]
 95%|█████████▌| 715/750 [1:14:39<03:28,  5.96s/it]
                                                   
{'loss': 0.1126, 'grad_norm': 0.18450631201267242, 'learning_rate': 8.967696494617803e-05, 'epoch': 4.77}
+
 95%|█████████▌| 715/750 [1:14:39<03:28,  5.96s/it]
 95%|█████████▌| 716/750 [1:14:45<03:22,  5.96s/it]
                                                   
{'loss': 0.1183, 'grad_norm': 0.18953324854373932, 'learning_rate': 8.961431948624752e-05, 'epoch': 4.77}
+
 95%|█████████▌| 716/750 [1:14:45<03:22,  5.96s/it]
 96%|█████████▌| 717/750 [1:14:51<03:16,  5.96s/it]
                                                   
{'loss': 0.1129, 'grad_norm': 0.17921291291713715, 'learning_rate': 8.955180512950356e-05, 'epoch': 4.78}
+
 96%|█████████▌| 717/750 [1:14:51<03:16,  5.96s/it]
 96%|█████████▌| 718/750 [1:14:57<03:10,  5.97s/it]
                                                   
{'loss': 0.108, 'grad_norm': 0.18071863055229187, 'learning_rate': 8.948942141929963e-05, 'epoch': 4.79}
+
 96%|█████████▌| 718/750 [1:14:57<03:10,  5.97s/it]
 96%|█████████▌| 719/750 [1:15:03<03:05,  5.97s/it]
                                                   
{'loss': 0.1286, 'grad_norm': 0.20430976152420044, 'learning_rate': 8.942716790121283e-05, 'epoch': 4.79}
+
 96%|█████████▌| 719/750 [1:15:03<03:05,  5.97s/it]
 96%|█████████▌| 720/750 [1:15:09<02:59,  5.97s/it]
                                                   
{'loss': 0.1151, 'grad_norm': 0.18577136099338531, 'learning_rate': 8.936504412303008e-05, 'epoch': 4.8}
+
 96%|█████████▌| 720/750 [1:15:09<02:59,  5.97s/it]
 96%|█████████▌| 721/750 [1:15:15<02:53,  5.98s/it]
                                                   
{'loss': 0.1387, 'grad_norm': 0.19394683837890625, 'learning_rate': 8.93030496347342e-05, 'epoch': 4.81}
+
 96%|█████████▌| 721/750 [1:15:15<02:53,  5.98s/it]
 96%|█████████▋| 722/750 [1:15:21<02:47,  5.98s/it]
                                                   
{'loss': 0.1254, 'grad_norm': 0.19409403204917908, 'learning_rate': 8.924118398849037e-05, 'epoch': 4.81}
+
 96%|█████████▋| 722/750 [1:15:21<02:47,  5.98s/it]
 96%|█████████▋| 723/750 [1:15:27<02:41,  5.97s/it]
                                                   
{'loss': 0.15, 'grad_norm': 0.19994400441646576, 'learning_rate': 8.917944673863242e-05, 'epoch': 4.82}
+
 96%|█████████▋| 723/750 [1:15:27<02:41,  5.97s/it]
 97%|█████████▋| 724/750 [1:15:33<02:34,  5.95s/it]
                                                   
{'loss': 0.1187, 'grad_norm': 0.19855041801929474, 'learning_rate': 8.911783744164937e-05, 'epoch': 4.83}
+
 97%|█████████▋| 724/750 [1:15:33<02:34,  5.95s/it]
 97%|█████████▋| 725/750 [1:15:39<02:28,  5.94s/it]
                                                   
{'loss': 0.12, 'grad_norm': 0.20143508911132812, 'learning_rate': 8.905635565617214e-05, 'epoch': 4.83}
+
 97%|█████████▋| 725/750 [1:15:39<02:28,  5.94s/it]
 97%|█████████▋| 726/750 [1:15:45<02:22,  5.94s/it]
                                                   
{'loss': 0.1274, 'grad_norm': 0.196522518992424, 'learning_rate': 8.899500094296009e-05, 'epoch': 4.84}
+
 97%|█████████▋| 726/750 [1:15:45<02:22,  5.94s/it]
 97%|█████████▋| 727/750 [1:15:51<02:16,  5.95s/it]
                                                   
{'loss': 0.1161, 'grad_norm': 0.18777668476104736, 'learning_rate': 8.893377286488804e-05, 'epoch': 4.85}
+
 97%|█████████▋| 727/750 [1:15:51<02:16,  5.95s/it]
 97%|█████████▋| 728/750 [1:15:57<02:12,  6.00s/it]
                                                   
{'loss': 0.1204, 'grad_norm': 0.1800437867641449, 'learning_rate': 8.887267098693303e-05, 'epoch': 4.85}
+
 97%|█████████▋| 728/750 [1:15:57<02:12,  6.00s/it]
 97%|█████████▋| 729/750 [1:16:03<02:05,  5.99s/it]
                                                   
{'loss': 0.1282, 'grad_norm': 0.20204044878482819, 'learning_rate': 8.881169487616148e-05, 'epoch': 4.86}
+
 97%|█████████▋| 729/750 [1:16:03<02:05,  5.99s/it]
 97%|█████████▋| 730/750 [1:16:09<01:59,  5.96s/it]
                                                   
{'loss': 0.1117, 'grad_norm': 0.20082896947860718, 'learning_rate': 8.875084410171615e-05, 'epoch': 4.87}
+
 97%|█████████▋| 730/750 [1:16:09<01:59,  5.96s/it]
 97%|█████████▋| 731/750 [1:16:15<01:53,  5.98s/it]
                                                   
{'loss': 0.1343, 'grad_norm': 0.19596554338932037, 'learning_rate': 8.869011823480348e-05, 'epoch': 4.87}
+
 97%|█████████▋| 731/750 [1:16:15<01:53,  5.98s/it]
 98%|█████████▊| 732/750 [1:16:21<01:47,  5.97s/it]
                                                   
{'loss': 0.1305, 'grad_norm': 0.21354715526103973, 'learning_rate': 8.862951684868085e-05, 'epoch': 4.88}
+
 98%|█████████▊| 732/750 [1:16:21<01:47,  5.97s/it]
 98%|█████████▊| 733/750 [1:16:27<01:41,  5.96s/it]
                                                   
{'loss': 0.1263, 'grad_norm': 0.1988142877817154, 'learning_rate': 8.856903951864397e-05, 'epoch': 4.89}
+
 98%|█████████▊| 733/750 [1:16:27<01:41,  5.96s/it]
 98%|█████████▊| 734/750 [1:16:33<01:35,  5.98s/it]
                                                   
{'loss': 0.1193, 'grad_norm': 0.19559569656848907, 'learning_rate': 8.85086858220144e-05, 'epoch': 4.89}
+
 98%|█████████▊| 734/750 [1:16:33<01:35,  5.98s/it]
 98%|█████████▊| 735/750 [1:16:39<01:29,  5.97s/it]
                                                   
{'loss': 0.1301, 'grad_norm': 0.20791786909103394, 'learning_rate': 8.844845533812719e-05, 'epoch': 4.9}
+
 98%|█████████▊| 735/750 [1:16:39<01:29,  5.97s/it]
 98%|█████████▊| 736/750 [1:16:44<01:23,  5.95s/it]
                                                   
{'loss': 0.1327, 'grad_norm': 0.20150598883628845, 'learning_rate': 8.838834764831844e-05, 'epoch': 4.91}
+
 98%|█████████▊| 736/750 [1:16:44<01:23,  5.95s/it]
 98%|█████████▊| 737/750 [1:16:50<01:17,  5.94s/it]
                                                   
{'loss': 0.1253, 'grad_norm': 0.1925792694091797, 'learning_rate': 8.83283623359132e-05, 'epoch': 4.91}
+
 98%|█████████▊| 737/750 [1:16:50<01:17,  5.94s/it]
 98%|█████████▊| 738/750 [1:16:56<01:11,  5.92s/it]
                                                   
{'loss': 0.1123, 'grad_norm': 0.19202423095703125, 'learning_rate': 8.826849898621327e-05, 'epoch': 4.92}
+
 98%|█████████▊| 738/750 [1:16:56<01:11,  5.92s/it]
 99%|█████████▊| 739/750 [1:17:02<01:05,  5.92s/it]
                                                   
{'loss': 0.1194, 'grad_norm': 0.21489877998828888, 'learning_rate': 8.82087571864852e-05, 'epoch': 4.93}
+
 99%|█████████▊| 739/750 [1:17:02<01:05,  5.92s/it]
 99%|█████████▊| 740/750 [1:17:08<00:59,  5.90s/it]
                                                   
{'loss': 0.1247, 'grad_norm': 0.19145846366882324, 'learning_rate': 8.814913652594829e-05, 'epoch': 4.93}
+
 99%|█████████▊| 740/750 [1:17:08<00:59,  5.90s/it]
 99%|█████████▉| 741/750 [1:17:14<00:52,  5.88s/it]
                                                   
{'loss': 0.14, 'grad_norm': 0.21258756518363953, 'learning_rate': 8.808963659576277e-05, 'epoch': 4.94}
+
 99%|█████████▉| 741/750 [1:17:14<00:52,  5.88s/it]
 99%|█████████▉| 742/750 [1:17:20<00:46,  5.87s/it]
                                                   
{'loss': 0.1338, 'grad_norm': 0.20081067085266113, 'learning_rate': 8.803025698901805e-05, 'epoch': 4.95}
+
 99%|█████████▉| 742/750 [1:17:20<00:46,  5.87s/it]
 99%|█████████▉| 743/750 [1:17:26<00:41,  5.89s/it]
                                                   
{'loss': 0.1243, 'grad_norm': 0.20036664605140686, 'learning_rate': 8.797099730072091e-05, 'epoch': 4.95}
+
 99%|█████████▉| 743/750 [1:17:26<00:41,  5.89s/it]
 99%|█████████▉| 744/750 [1:17:31<00:35,  5.88s/it]
                                                   
{'loss': 0.1208, 'grad_norm': 0.19513240456581116, 'learning_rate': 8.791185712778405e-05, 'epoch': 4.96}
+
 99%|█████████▉| 744/750 [1:17:32<00:35,  5.88s/it]
 99%|█████████▉| 745/750 [1:17:37<00:29,  5.88s/it]
                                                   
{'loss': 0.1249, 'grad_norm': 0.2070721685886383, 'learning_rate': 8.785283606901446e-05, 'epoch': 4.97}
+
 99%|█████████▉| 745/750 [1:17:37<00:29,  5.88s/it]
 99%|█████████▉| 746/750 [1:17:43<00:23,  5.87s/it]
                                                   
{'loss': 0.152, 'grad_norm': 0.2183961719274521, 'learning_rate': 8.779393372510207e-05, 'epoch': 4.97}
+
 99%|█████████▉| 746/750 [1:17:43<00:23,  5.87s/it]
100%|█████████▉| 747/750 [1:17:49<00:17,  5.92s/it]
                                                   
{'loss': 0.1179, 'grad_norm': 0.20051303505897522, 'learning_rate': 8.773514969860834e-05, 'epoch': 4.98}
+
100%|█████████▉| 747/750 [1:17:49<00:17,  5.92s/it]
100%|█████████▉| 748/750 [1:17:55<00:11,  5.93s/it]
                                                   
{'loss': 0.128, 'grad_norm': 0.18662500381469727, 'learning_rate': 8.767648359395506e-05, 'epoch': 4.99}
+
100%|█████████▉| 748/750 [1:17:55<00:11,  5.93s/it]
100%|█████████▉| 749/750 [1:18:01<00:05,  5.96s/it]
                                                   
{'loss': 0.1166, 'grad_norm': 0.18965964019298553, 'learning_rate': 8.761793501741308e-05, 'epoch': 4.99}
+
100%|█████████▉| 749/750 [1:18:01<00:05,  5.96s/it]
100%|██████████| 750/750 [1:18:09<00:00,  6.58s/it]
                                                   
{'loss': 0.1321, 'grad_norm': 0.2086346298456192, 'learning_rate': 8.755950357709131e-05, 'epoch': 5.0}
+
100%|██████████| 750/750 [1:18:09<00:00,  6.58s/it][INFO|trainer.py:3515] 2025-06-27 01:07:06,408 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-750
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da162-7a76ce6814b2933b0506bdc8;e31c84cb-902c-45c7-a8c4-68586d16784b)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-27 01:07:06,802 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-750/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-27 01:07:06,803 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-750/special_tokens_map.json
+[2025-06-27 01:07:11,102] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Checkpoint global_step750 is begin to save!
+[2025-06-27 01:07:11,124] [INFO] [logging.py:107:log_dist] [Rank 0] Saving model checkpoint: ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-750/global_step750/mp_rank_00_model_states.pt
+[INFO|trainer.py:3607] 2025-06-27 01:07:11,253 >> Deleting older checkpoint [outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-600] due to args.save_total_limit
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da167-0da98d195b602ddd1da097f1;5b85b711-8b7f-4848-99fb-8db60242d425)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da167-171c83702e81a00c50282adb;20a3a0db-c780-4f86-a174-de59c54540bd)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da167-401573c35c22a55a47e63ebe;8f6274b5-a237-441b-9042-e65a0e5353e5)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+[INFO|trainer.py:2406] 2025-06-27 01:07:11,646 >> 
+
+Training completed. Do not forget to share your model on huggingface.co/models =)
+
+
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da167-71037e814a30726033b63379;07ef2bfb-df7d-4317-a41d-e71761b82666)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+[INFO|trainer.py:2644] 2025-06-27 01:07:11,978 >> Loading best model from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-400 (score: 0.20738115906715393).
+[INFO|deepspeed.py:431] 2025-06-27 01:07:11,979 >> Attempting to resume from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-400
+[2025-06-27 01:07:11,980] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Begin Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt...
+[2025-06-27 01:07:11,999] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] End Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt...
+[2025-06-27 01:07:12,000] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Begin Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt...
+[2025-06-27 01:07:12,014] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] End Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt...
+[2025-06-27 01:07:12,045] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Begin Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt...
+[2025-06-27 01:07:12,060] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] End Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt...
+[2025-06-27 01:07:12,060] [INFO] [engine.py:3277:_get_all_zero_checkpoint_state_dicts] successfully read 4 ZeRO state_dicts for rank 0
+[2025-06-27 01:07:12,069] [INFO] [engine.py:3227:_load_zero_checkpoint] loading 4 zero partition checkpoints for rank 0
+
                                                   
{'train_runtime': 4707.1456, 'train_samples_per_second': 15.933, 'train_steps_per_second': 0.159, 'train_loss': 0.17488925837477048, 'epoch': 5.0}
+
100%|██████████| 750/750 [1:18:25<00:00,  6.58s/it][INFO|trainer.py:2447] 2025-06-27 01:07:12,078 >> Deleting older checkpoint [outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/checkpoint-750] due to args.save_total_limit
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da168-19b7bd84391e677018665b40;8110803f-ebfc-4ef4-a01c-3cbb7ad128b2)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da168-2de248032402e78410e9eb77;abd1291e-07b4-4651-9d36-031537ee247a)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da168-5add01581039cf7922a03e78;86e3433b-5446-4523-a1e9-5a8830faaedd)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da168-4623087c69fb423053ad278d;5951f658-6b53-4066-8b5c-8d199dfd3509)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+
100%|██████████| 750/750 [1:18:25<00:00,  6.27s/it]
+[INFO|trainer.py:3515] 2025-06-27 01:07:21,803 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da171-649b365b75223cc009186264;2de77f84-2317-42a3-9e2b-4685dd5ba448)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-27 01:07:22,131 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-27 01:07:22,132 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/special_tokens_map.json
+***** train metrics *****
+  epoch                    =          5.0
+  total_flos               = 6234656320GF
+  train_loss               =       0.1749
+  train_runtime            =   1:18:27.14
+  train_samples            =        15000
+  train_samples_per_second =       15.933
+  train_steps_per_second   =        0.159
+06/27/2025 01:07:23 - INFO - __main__ - *** Evaluate ***
+[INFO|trainer.py:3831] 2025-06-27 01:07:23,242 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-27 01:07:23,242 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-27 01:07:23,242 >>   Batch size = 25
+
  0%|          | 0/10 [00:00<?, ?it/s]
 20%|██        | 2/10 [00:01<00:07,  1.08it/s]
 30%|███       | 3/10 [00:03<00:09,  1.30s/it]
 40%|████      | 4/10 [00:05<00:08,  1.49s/it]
 50%|█████     | 5/10 [00:07<00:08,  1.62s/it]
 60%|██████    | 6/10 [00:09<00:06,  1.69s/it]
 70%|███████   | 7/10 [00:11<00:05,  1.73s/it]
 80%|████████  | 8/10 [00:12<00:03,  1.76s/it]
 90%|█████████ | 9/10 [00:14<00:01,  1.77s/it]
100%|██████████| 10/10 [00:18<00:00,  2.44s/it]
100%|██████████| 10/10 [00:24<00:00,  2.43s/it]
+***** eval metrics *****
+  epoch                   =        5.0
+  eval_accuracy           =     0.2232
+  eval_loss               =     0.2074
+  eval_runtime            = 0:00:37.33
+  eval_samples            =       1000
+  eval_samples_per_second =     26.788
+  eval_steps_per_second   =      0.268
+  perplexity              =     1.2305
+[INFO|modelcard.py:449] 2025-06-27 01:08:00,855 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.22317733268197362}]}
+wandb: 
+wandb: 🚀 View run ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/bn/baseline/data_15000_1000/ at: https://wandb.ai/indic-encoder/midalign/runs/s645lnzf
+wandb: Find logs at: wandb/run-20250626_234845-s645lnzf/logs
+[rank0]:[W627 01:08:06.152459494 ProcessGroupNCCL.cpp:1479] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
diff --git a/bn/baseline/data_15000_1000/train_results.json b/bn/baseline/data_15000_1000/train_results.json
new file mode 100644
index 0000000000000000000000000000000000000000..7a0bc1690432c4b12caeda3077be139a93e742cb
--- /dev/null
+++ b/bn/baseline/data_15000_1000/train_results.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d1d9b636207a2b575e6efa4dcaba32228031d6884143a9811f24b73d1ef4a151
+size 237
diff --git a/bn/baseline/data_15000_1000/trainer_state.json b/bn/baseline/data_15000_1000/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..aaebe45c0eae7efc321dd644f080f7a64071572c
--- /dev/null
+++ b/bn/baseline/data_15000_1000/trainer_state.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:809c1f409636297cdf9dea1fcdbb52a9204d19b9f63069dccb0dcada4c179104
+size 129343
diff --git a/bn/baseline/data_15000_1000/training_args.bin b/bn/baseline/data_15000_1000/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..75e2af92638c9ebdd3a0bab1aeafcb8d42d32cdb
--- /dev/null
+++ b/bn/baseline/data_15000_1000/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3acc7c6d4109909cd8d0a6012eabedc6936794f718ccb8e9dda270d768c9160
+size 7761
diff --git a/en/baseline/data_15000_1000/README.md b/en/baseline/data_15000_1000/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..223512faebf69fc1567077e64875dbca1c0279bc
--- /dev/null
+++ b/en/baseline/data_15000_1000/README.md
@@ -0,0 +1,70 @@
+---
+license: llama3.1
+base_model: meta-llama/Llama-3.1-8B-Instruct
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+library_name: peft
+model-index:
+- name: data_15000_1000
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# data_15000_1000
+
+This model is a fine-tuned version of [meta-llama/Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.2748
+- Accuracy: 0.4615
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0005
+- train_batch_size: 25
+- eval_batch_size: 25
+- seed: 1
+- distributed_type: multi-GPU
+- num_devices: 4
+- total_train_batch_size: 100
+- total_eval_batch_size: 100
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: inverse_sqrt
+- lr_scheduler_warmup_ratio: 0.03
+- num_epochs: 5.0
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Accuracy |
+|:-------------:|:------:|:----:|:---------------:|:--------:|
+| No log        | 0      | 0    | 0.5910          | 0.4539   |
+| 0.2403        | 1.3333 | 200  | 0.2748          | 0.4615   |
+| 0.1768        | 2.6667 | 400  | 0.2830          | 0.4615   |
+| 0.1374        | 4.0    | 600  | 0.3029          | 0.4614   |
+
+
+### Framework versions
+
+- PEFT 0.15.2
+- Transformers 4.44.0.dev0
+- Pytorch 2.7.1+cu126
+- Datasets 3.6.0
+- Tokenizers 0.19.1
\ No newline at end of file
diff --git a/en/baseline/data_15000_1000/adapter_config.json b/en/baseline/data_15000_1000/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..98f15c0f5c9b05f961cc1d5ccdef09b5966455d3
--- /dev/null
+++ b/en/baseline/data_15000_1000/adapter_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e3d277b14ce5c2397fd85d3b81d78f059e80359319b0dbbc4524175d7e26be2
+size 863
diff --git a/en/baseline/data_15000_1000/adapter_model.safetensors b/en/baseline/data_15000_1000/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..415eaa35201d4ac55bf2e917b21c17ea56430749
--- /dev/null
+++ b/en/baseline/data_15000_1000/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa8cf150f379c3a6d83d75032a403e0fde84e04c81403d9ac91ac4a27f4915a3
+size 42002584
diff --git a/en/baseline/data_15000_1000/adapter_model/README.md b/en/baseline/data_15000_1000/adapter_model/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0d31128190920e45b61115944d16e773c2ec94c3
--- /dev/null
+++ b/en/baseline/data_15000_1000/adapter_model/README.md
@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/en/baseline/data_15000_1000/adapter_model/adapter_config.json b/en/baseline/data_15000_1000/adapter_model/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..9c683d80a0a7cdb896e906c571b87d7ab4476404
--- /dev/null
+++ b/en/baseline/data_15000_1000/adapter_model/adapter_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8011dcc5caa2c08c0faf23d1480636aff6bc68c176c8f7b0d34a57bc475fcb0
+size 863
diff --git a/en/baseline/data_15000_1000/adapter_model/adapter_model.safetensors b/en/baseline/data_15000_1000/adapter_model/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..415eaa35201d4ac55bf2e917b21c17ea56430749
--- /dev/null
+++ b/en/baseline/data_15000_1000/adapter_model/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa8cf150f379c3a6d83d75032a403e0fde84e04c81403d9ac91ac4a27f4915a3
+size 42002584
diff --git a/en/baseline/data_15000_1000/all_results.json b/en/baseline/data_15000_1000/all_results.json
new file mode 100644
index 0000000000000000000000000000000000000000..06721fe484a383613b8c3881997a2c4d52d9f0e1
--- /dev/null
+++ b/en/baseline/data_15000_1000/all_results.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57cb88f0db39b3e64fe969c24cce860f26f118857822275f9403f51a6f56e405
+size 485
diff --git a/en/baseline/data_15000_1000/checkpoint-200/README.md b/en/baseline/data_15000_1000/checkpoint-200/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0d31128190920e45b61115944d16e773c2ec94c3
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/README.md
@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/en/baseline/data_15000_1000/checkpoint-200/adapter_config.json b/en/baseline/data_15000_1000/checkpoint-200/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..98f15c0f5c9b05f961cc1d5ccdef09b5966455d3
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/adapter_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e3d277b14ce5c2397fd85d3b81d78f059e80359319b0dbbc4524175d7e26be2
+size 863
diff --git a/en/baseline/data_15000_1000/checkpoint-200/adapter_model.safetensors b/en/baseline/data_15000_1000/checkpoint-200/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..415eaa35201d4ac55bf2e917b21c17ea56430749
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa8cf150f379c3a6d83d75032a403e0fde84e04c81403d9ac91ac4a27f4915a3
+size 42002584
diff --git a/en/baseline/data_15000_1000/checkpoint-200/adapter_model/README.md b/en/baseline/data_15000_1000/checkpoint-200/adapter_model/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0d31128190920e45b61115944d16e773c2ec94c3
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/adapter_model/README.md
@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/en/baseline/data_15000_1000/checkpoint-200/adapter_model/adapter_config.json b/en/baseline/data_15000_1000/checkpoint-200/adapter_model/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..bc9a4c6dedf77cc69e546c2dd963ff372acd6c2c
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/adapter_model/adapter_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6870dfdbba475aeda09684387a4704ab0c48e8b8f997bdbee0485b8c2a5cc13c
+size 863
diff --git a/en/baseline/data_15000_1000/checkpoint-200/adapter_model/adapter_model.safetensors b/en/baseline/data_15000_1000/checkpoint-200/adapter_model/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..415eaa35201d4ac55bf2e917b21c17ea56430749
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/adapter_model/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa8cf150f379c3a6d83d75032a403e0fde84e04c81403d9ac91ac4a27f4915a3
+size 42002584
diff --git a/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt b/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..455736452174777899dc789db233da3a5a992026
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:617c6d383e372f898784e12a7ea1b7e3fb15511f9db83a78de33af4b75e75b59
+size 62930469
diff --git a/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt b/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..f547b9c8a34c53024f7c2fd2ed1509da6c74cdf1
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce9de7b1aea78bd05425d72424778e88f2d8e54a9e63fdf4b663dfcc0d511b28
+size 62930533
diff --git a/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt b/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..3f90dcdf4f1e96b0e493d8dee94353394e66da68
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78ced5b3d36bc56998fe085f7f321c99d64e88ae7db3ac416de98bd1345aaa17
+size 62930597
diff --git a/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt b/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..78b669a9089ecfeda02df9cea4383b33f238eadc
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:182a0c6265e9b311a10a95134c1815d6415c78eaf19e9b90b6402164718dcd6b
+size 62930597
diff --git a/en/baseline/data_15000_1000/checkpoint-200/global_step200/mp_rank_00_model_states.pt b/en/baseline/data_15000_1000/checkpoint-200/global_step200/mp_rank_00_model_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..0127d510372a5d53f0d1451d90ba1f879553acb4
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/global_step200/mp_rank_00_model_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9bb06b7c824218354747ff326d2583a6ee99557b74146b472f769eb9355fdfa1
+size 42291289
diff --git a/en/baseline/data_15000_1000/checkpoint-200/latest b/en/baseline/data_15000_1000/checkpoint-200/latest
new file mode 100644
index 0000000000000000000000000000000000000000..753e24e10f3a2489150f458205cf759fd8b6081f
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/latest
@@ -0,0 +1 @@
+global_step200
\ No newline at end of file
diff --git a/en/baseline/data_15000_1000/checkpoint-200/rng_state_0.pth b/en/baseline/data_15000_1000/checkpoint-200/rng_state_0.pth
new file mode 100644
index 0000000000000000000000000000000000000000..b60032f5937d03d79cc2faac8de98c3ff048908a
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/rng_state_0.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b994fbe4bca16ade24247fd6bc06dbd109fe47e1e59efffd90abb94d574ccf65
+size 15429
diff --git a/en/baseline/data_15000_1000/checkpoint-200/rng_state_1.pth b/en/baseline/data_15000_1000/checkpoint-200/rng_state_1.pth
new file mode 100644
index 0000000000000000000000000000000000000000..cf40507674edcf7f4355f9cd632f9bd9ccd26bcf
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/rng_state_1.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d58c8c81036d393c97609860292dd1fb81b87558c220690c4e3a501cf809cea
+size 15429
diff --git a/en/baseline/data_15000_1000/checkpoint-200/rng_state_2.pth b/en/baseline/data_15000_1000/checkpoint-200/rng_state_2.pth
new file mode 100644
index 0000000000000000000000000000000000000000..93a60a38415439c3c63a41356c771c978f6ca074
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/rng_state_2.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc8dbf2e90683e25732ecf0e3253eea7493ebe05ee381fe77ca8a498f2724452
+size 15429
diff --git a/en/baseline/data_15000_1000/checkpoint-200/rng_state_3.pth b/en/baseline/data_15000_1000/checkpoint-200/rng_state_3.pth
new file mode 100644
index 0000000000000000000000000000000000000000..8456829c95d3005151bc3821a1920bb705fef27c
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/rng_state_3.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd5526116d019a2690b6d66ff864a8aef6744dddcea0a767f442fa7bb7e19c35
+size 15429
diff --git a/en/baseline/data_15000_1000/checkpoint-200/special_tokens_map.json b/en/baseline/data_15000_1000/checkpoint-200/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..4ed2bd8240878a7a0d4fd2c60cdc89f6d7a5f1e1
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/special_tokens_map.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83da1082df286d75a2984dc06ec439f4febc3d862ac55d199402e5d345f5773a
+size 372
diff --git a/en/baseline/data_15000_1000/checkpoint-200/tokenizer.json b/en/baseline/data_15000_1000/checkpoint-200/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..66cd9d7e0daec95eb10d16a63c615637dbbb7304
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79e3e522635f3171300913bb421464a87de6222182a0570b9b2ccba2a964b2b4
+size 9085657
diff --git a/en/baseline/data_15000_1000/checkpoint-200/tokenizer_config.json b/en/baseline/data_15000_1000/checkpoint-200/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..7144ada11807e90b92529f17434f8d01915c3dff
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/tokenizer_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d967a51bb800d3e471ea23dd3f7b054b136600238bbbb40612b8b96b0370746e
+size 55427
diff --git a/en/baseline/data_15000_1000/checkpoint-200/trainer_state.json b/en/baseline/data_15000_1000/checkpoint-200/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..7d9f9734f415f59045f7a9bb03c971d4f7aacb2f
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/trainer_state.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3efcfd09a1a939188162b62a312b7c9e568f02c56e626a9b87170763f3d2f72
+size 35399
diff --git a/en/baseline/data_15000_1000/checkpoint-200/training_args.bin b/en/baseline/data_15000_1000/checkpoint-200/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..faffedb31b34c9635c09771f47b340cecfa5f06a
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:919f8d1e8774d9c124d170364fd95fc7a9b43c2f0cfecf93b3d0c961ad69acb1
+size 7761
diff --git a/en/baseline/data_15000_1000/checkpoint-200/zero_to_fp32.py b/en/baseline/data_15000_1000/checkpoint-200/zero_to_fp32.py
new file mode 100755
index 0000000000000000000000000000000000000000..0e759146cadd92ddfefab3680146c2bd6a2b5c04
--- /dev/null
+++ b/en/baseline/data_15000_1000/checkpoint-200/zero_to_fp32.py
@@ -0,0 +1,760 @@
+#!/usr/bin/env python
+
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: Apache-2.0
+
+# DeepSpeed Team
+
+# This script extracts fp32 consolidated weights from a zero 1, 2 and 3 DeepSpeed checkpoints. It gets
+# copied into the top level checkpoint dir, so the user can easily do the conversion at any point in
+# the future. Once extracted, the weights don't require DeepSpeed and can be used in any
+# application.
+#
+# example:
+#   python zero_to_fp32.py . output_dir/
+#   or
+#   python zero_to_fp32.py . output_dir/ --safe_serialization
+
+import argparse
+import torch
+import glob
+import math
+import os
+import re
+import gc
+import json
+import numpy as np
+from tqdm import tqdm
+from collections import OrderedDict
+from dataclasses import dataclass
+
+# while this script doesn't use deepspeed to recover data, since the checkpoints are pickled with
+# DeepSpeed data structures it has to be available in the current python environment.
+from deepspeed.utils import logger
+from deepspeed.checkpoint.constants import (DS_VERSION, OPTIMIZER_STATE_DICT, SINGLE_PARTITION_OF_FP32_GROUPS,
+                                            FP32_FLAT_GROUPS, ZERO_STAGE, PARTITION_COUNT, PARAM_SHAPES, BUFFER_NAMES,
+                                            FROZEN_PARAM_SHAPES, FROZEN_PARAM_FRAGMENTS)
+
+
+@dataclass
+class zero_model_state:
+    buffers: dict()
+    param_shapes: dict()
+    shared_params: list
+    ds_version: int
+    frozen_param_shapes: dict()
+    frozen_param_fragments: dict()
+
+
+debug = 0
+
+# load to cpu
+device = torch.device('cpu')
+
+
+def atoi(text):
+    return int(text) if text.isdigit() else text
+
+
+def natural_keys(text):
+    '''
+    alist.sort(key=natural_keys) sorts in human order
+    http://nedbatchelder.com/blog/200712/human_sorting.html
+    (See Toothy's implementation in the comments)
+    '''
+    return [atoi(c) for c in re.split(r'(\d+)', text)]
+
+
+def get_model_state_file(checkpoint_dir, zero_stage):
+    if not os.path.isdir(checkpoint_dir):
+        raise FileNotFoundError(f"Directory '{checkpoint_dir}' doesn't exist")
+
+    # there should be only one file
+    if zero_stage <= 2:
+        file = os.path.join(checkpoint_dir, "mp_rank_00_model_states.pt")
+    elif zero_stage == 3:
+        file = os.path.join(checkpoint_dir, "zero_pp_rank_0_mp_rank_00_model_states.pt")
+
+    if not os.path.exists(file):
+        raise FileNotFoundError(f"can't find model states file at '{file}'")
+
+    return file
+
+
+def get_checkpoint_files(checkpoint_dir, glob_pattern):
+    # XXX: need to test that this simple glob rule works for multi-node setup too
+    ckpt_files = sorted(glob.glob(os.path.join(checkpoint_dir, glob_pattern)), key=natural_keys)
+
+    if len(ckpt_files) == 0:
+        raise FileNotFoundError(f"can't find {glob_pattern} files in directory '{checkpoint_dir}'")
+
+    return ckpt_files
+
+
+def get_optim_files(checkpoint_dir):
+    return get_checkpoint_files(checkpoint_dir, "*_optim_states.pt")
+
+
+def get_model_state_files(checkpoint_dir):
+    return get_checkpoint_files(checkpoint_dir, "*_model_states.pt")
+
+
+def parse_model_states(files):
+    zero_model_states = []
+    for file in files:
+        state_dict = torch.load(file, map_location=device, weights_only=False)
+
+        if BUFFER_NAMES not in state_dict:
+            raise ValueError(f"{file} is not a model state checkpoint")
+        buffer_names = state_dict[BUFFER_NAMES]
+        if debug:
+            print("Found buffers:", buffer_names)
+
+        # recover just the buffers while restoring them to fp32 if they were saved in fp16
+        buffers = {k: v.float() for k, v in state_dict["module"].items() if k in buffer_names}
+        param_shapes = state_dict[PARAM_SHAPES]
+
+        # collect parameters that are included in param_shapes
+        param_names = []
+        for s in param_shapes:
+            for name in s.keys():
+                param_names.append(name)
+
+        # update with frozen parameters
+        frozen_param_shapes = state_dict.get(FROZEN_PARAM_SHAPES, None)
+        if frozen_param_shapes is not None:
+            if debug:
+                print(f"Found frozen_param_shapes: {frozen_param_shapes}")
+            param_names += list(frozen_param_shapes.keys())
+
+        # handle shared params
+        shared_params = [[k, v] for k, v in state_dict["shared_params"].items()]
+
+        ds_version = state_dict.get(DS_VERSION, None)
+
+        frozen_param_fragments = state_dict.get(FROZEN_PARAM_FRAGMENTS, None)
+
+        z_model_state = zero_model_state(buffers=buffers,
+                                         param_shapes=param_shapes,
+                                         shared_params=shared_params,
+                                         ds_version=ds_version,
+                                         frozen_param_shapes=frozen_param_shapes,
+                                         frozen_param_fragments=frozen_param_fragments)
+        zero_model_states.append(z_model_state)
+
+    return zero_model_states
+
+
+def parse_optim_states(files, ds_checkpoint_dir):
+    total_files = len(files)
+    state_dicts = []
+    for f in tqdm(files, desc='Loading checkpoint shards'):
+        state_dict = torch.load(f, map_location=device, mmap=True, weights_only=False)
+        # immediately discard the potentially huge 2 optimizer states as we only care for fp32 master weights
+        # and also handle the case where it was already removed by another helper script
+        state_dict["optimizer_state_dict"].pop("optimizer_state_dict", None)
+        state_dicts.append(state_dict)
+
+    if not ZERO_STAGE in state_dicts[0][OPTIMIZER_STATE_DICT]:
+        raise ValueError(f"{files[0]} is not a zero checkpoint")
+    zero_stage = state_dicts[0][OPTIMIZER_STATE_DICT][ZERO_STAGE]
+    world_size = state_dicts[0][OPTIMIZER_STATE_DICT][PARTITION_COUNT]
+
+    # For ZeRO-2 each param group can have different partition_count as data parallelism for expert
+    # parameters can be different from data parallelism for non-expert parameters. So we can just
+    # use the max of the partition_count to get the dp world_size.
+
+    if type(world_size) is list:
+        world_size = max(world_size)
+
+    if world_size != total_files:
+        raise ValueError(
+            f"Expected {world_size} of '*_optim_states.pt' under '{ds_checkpoint_dir}' but found {total_files} files. "
+            "Possibly due to an overwrite of an old checkpoint, or a checkpoint didn't get saved by one or more processes."
+        )
+
+    # the groups are named differently in each stage
+    if zero_stage <= 2:
+        fp32_groups_key = SINGLE_PARTITION_OF_FP32_GROUPS
+    elif zero_stage == 3:
+        fp32_groups_key = FP32_FLAT_GROUPS
+    else:
+        raise ValueError(f"unknown zero stage {zero_stage}")
+
+    fp32_flat_groups = [state_dicts[i][OPTIMIZER_STATE_DICT][fp32_groups_key] for i in range(len(state_dicts))]
+    return zero_stage, world_size, fp32_flat_groups
+
+
+def _get_fp32_state_dict_from_zero_checkpoint(ds_checkpoint_dir, exclude_frozen_parameters):
+    """
+    Returns fp32 state_dict reconstructed from ds checkpoint
+
+    Args:
+        - ``ds_checkpoint_dir``: path to the deepspeed checkpoint folder (where the optimizer files are)
+
+    """
+    print(f"Processing zero checkpoint '{ds_checkpoint_dir}'")
+
+    optim_files = get_optim_files(ds_checkpoint_dir)
+    zero_stage, world_size, fp32_flat_groups = parse_optim_states(optim_files, ds_checkpoint_dir)
+    print(f"Detected checkpoint of type zero stage {zero_stage}, world_size: {world_size}")
+
+    model_files = get_model_state_files(ds_checkpoint_dir)
+
+    zero_model_states = parse_model_states(model_files)
+    print(f'Parsing checkpoint created by deepspeed=={zero_model_states[0].ds_version}')
+
+    if zero_stage <= 2:
+        return _get_fp32_state_dict_from_zero2_checkpoint(world_size, fp32_flat_groups, zero_model_states,
+                                                          exclude_frozen_parameters)
+    elif zero_stage == 3:
+        return _get_fp32_state_dict_from_zero3_checkpoint(world_size, fp32_flat_groups, zero_model_states,
+                                                          exclude_frozen_parameters)
+
+
+def _zero2_merge_frozen_params(state_dict, zero_model_states):
+    if zero_model_states[0].frozen_param_shapes is None or len(zero_model_states[0].frozen_param_shapes) == 0:
+        return
+
+    frozen_param_shapes = zero_model_states[0].frozen_param_shapes
+    frozen_param_fragments = zero_model_states[0].frozen_param_fragments
+
+    if debug:
+        num_elem = sum(s.numel() for s in frozen_param_shapes.values())
+        print(f'rank 0: {FROZEN_PARAM_SHAPES}.numel = {num_elem}')
+
+        wanted_params = len(frozen_param_shapes)
+        wanted_numel = sum(s.numel() for s in frozen_param_shapes.values())
+        avail_numel = sum([p.numel() for p in frozen_param_fragments.values()])
+        print(f'Frozen params: Have {avail_numel} numels to process.')
+        print(f'Frozen params: Need {wanted_numel} numels in {wanted_params} params')
+
+    total_params = 0
+    total_numel = 0
+    for name, shape in frozen_param_shapes.items():
+        total_params += 1
+        unpartitioned_numel = shape.numel()
+        total_numel += unpartitioned_numel
+
+        state_dict[name] = frozen_param_fragments[name]
+
+        if debug:
+            print(f"{name} full shape: {shape} unpartitioned numel {unpartitioned_numel} ")
+
+    print(f"Reconstructed Frozen fp32 state dict with {total_params} params {total_numel} elements")
+
+
+def _has_callable(obj, fn):
+    attr = getattr(obj, fn, None)
+    return callable(attr)
+
+
+def _zero2_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states):
+    param_shapes = zero_model_states[0].param_shapes
+
+    # Reconstruction protocol:
+    #
+    # XXX: document this
+
+    if debug:
+        for i in range(world_size):
+            for j in range(len(fp32_flat_groups[0])):
+                print(f"{FP32_FLAT_GROUPS}[{i}][{j}].shape={fp32_flat_groups[i][j].shape}")
+
+    # XXX: memory usage doubles here (zero2)
+    num_param_groups = len(fp32_flat_groups[0])
+    merged_single_partition_of_fp32_groups = []
+    for i in range(num_param_groups):
+        merged_partitions = [sd[i] for sd in fp32_flat_groups]
+        full_single_fp32_vector = torch.cat(merged_partitions, 0)
+        merged_single_partition_of_fp32_groups.append(full_single_fp32_vector)
+    avail_numel = sum(
+        [full_single_fp32_vector.numel() for full_single_fp32_vector in merged_single_partition_of_fp32_groups])
+
+    if debug:
+        wanted_params = sum([len(shapes) for shapes in param_shapes])
+        wanted_numel = sum([sum(shape.numel() for shape in shapes.values()) for shapes in param_shapes])
+        # not asserting if there is a mismatch due to possible padding
+        print(f"Have {avail_numel} numels to process.")
+        print(f"Need {wanted_numel} numels in {wanted_params} params.")
+
+    # params
+    # XXX: for huge models that can't fit into the host's RAM we will have to recode this to support
+    # out-of-core computing solution
+    total_numel = 0
+    total_params = 0
+    for shapes, full_single_fp32_vector in zip(param_shapes, merged_single_partition_of_fp32_groups):
+        offset = 0
+        avail_numel = full_single_fp32_vector.numel()
+        for name, shape in shapes.items():
+
+            unpartitioned_numel = shape.numel() if _has_callable(shape, 'numel') else math.prod(shape)
+            total_numel += unpartitioned_numel
+            total_params += 1
+
+            if debug:
+                print(f"{name} full shape: {shape} unpartitioned numel {unpartitioned_numel} ")
+            state_dict[name] = full_single_fp32_vector.narrow(0, offset, unpartitioned_numel).view(shape)
+            offset += unpartitioned_numel
+
+        # Z2 started to align to 2*world_size to improve nccl performance. Therefore both offset and
+        # avail_numel can differ by anywhere between 0..2*world_size. Due to two unrelated complex
+        # paddings performed in the code it's almost impossible to predict the exact numbers w/o the
+        # live optimizer object, so we are checking that the numbers are within the right range
+        align_to = 2 * world_size
+
+        def zero2_align(x):
+            return align_to * math.ceil(x / align_to)
+
+        if debug:
+            print(f"original offset={offset}, avail_numel={avail_numel}")
+
+        offset = zero2_align(offset)
+        avail_numel = zero2_align(avail_numel)
+
+        if debug:
+            print(f"aligned  offset={offset}, avail_numel={avail_numel}")
+
+        # Sanity check
+        if offset != avail_numel:
+            raise ValueError(f"consumed {offset} numels out of {avail_numel} - something is wrong")
+
+    print(f"Reconstructed fp32 state dict with {total_params} params {total_numel} elements")
+
+
+def _get_fp32_state_dict_from_zero2_checkpoint(world_size, fp32_flat_groups, zero_model_states,
+                                               exclude_frozen_parameters):
+    state_dict = OrderedDict()
+
+    # buffers
+    buffers = zero_model_states[0].buffers
+    state_dict.update(buffers)
+    if debug:
+        print(f"added {len(buffers)} buffers")
+
+    if not exclude_frozen_parameters:
+        _zero2_merge_frozen_params(state_dict, zero_model_states)
+
+    _zero2_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states)
+
+    # recover shared parameters
+    for pair in zero_model_states[0].shared_params:
+        if pair[1] in state_dict:
+            state_dict[pair[0]] = state_dict[pair[1]]
+
+    return state_dict
+
+
+def zero3_partitioned_param_info(unpartitioned_numel, world_size):
+    remainder = unpartitioned_numel % world_size
+    padding_numel = (world_size - remainder) if remainder else 0
+    partitioned_numel = math.ceil(unpartitioned_numel / world_size)
+    return partitioned_numel, padding_numel
+
+
+def _zero3_merge_frozen_params(state_dict, world_size, zero_model_states):
+    if zero_model_states[0].frozen_param_shapes is None or len(zero_model_states[0].frozen_param_shapes) == 0:
+        return
+
+    if debug:
+        for i in range(world_size):
+            num_elem = sum(s.numel() for s in zero_model_states[i].frozen_param_fragments.values())
+            print(f'rank {i}: {FROZEN_PARAM_SHAPES}.numel = {num_elem}')
+
+        frozen_param_shapes = zero_model_states[0].frozen_param_shapes
+        wanted_params = len(frozen_param_shapes)
+        wanted_numel = sum(s.numel() for s in frozen_param_shapes.values())
+        avail_numel = sum([p.numel() for p in zero_model_states[0].frozen_param_fragments.values()]) * world_size
+        print(f'Frozen params: Have {avail_numel} numels to process.')
+        print(f'Frozen params: Need {wanted_numel} numels in {wanted_params} params')
+
+    total_params = 0
+    total_numel = 0
+    for name, shape in zero_model_states[0].frozen_param_shapes.items():
+        total_params += 1
+        unpartitioned_numel = shape.numel()
+        total_numel += unpartitioned_numel
+
+        param_frags = tuple(model_state.frozen_param_fragments[name] for model_state in zero_model_states)
+        state_dict[name] = torch.cat(param_frags, 0).narrow(0, 0, unpartitioned_numel).view(shape)
+
+        partitioned_numel, partitioned_padding_numel = zero3_partitioned_param_info(unpartitioned_numel, world_size)
+
+        if debug:
+            print(
+                f"Frozen params: {total_params} {name} full shape: {shape} partition0 numel={partitioned_numel} partitioned_padding_numel={partitioned_padding_numel}"
+            )
+
+    print(f"Reconstructed Frozen fp32 state dict with {total_params} params {total_numel} elements")
+
+
+class GatheredTensor:
+    """
+    A pseudo tensor that collects partitioned weights.
+    It is more memory efficient when there are multiple groups.
+    """
+
+    def __init__(self, flat_groups, flat_groups_offset, offset, partitioned_numel, shape):
+        self.flat_groups = flat_groups
+        self.flat_groups_offset = flat_groups_offset
+        self.offset = offset
+        self.partitioned_numel = partitioned_numel
+        self.shape = shape
+        self.dtype = self.flat_groups[0][0].dtype
+
+    def contiguous(self):
+        """
+        Merge partitioned weights from flat_groups into a single tensor.
+        """
+        end_idx = self.offset + self.partitioned_numel
+        world_size = len(self.flat_groups)
+        pad_flat_param_chunks = []
+
+        for rank_i in range(world_size):
+            # for each rank, we need to collect weights from related group/groups
+            flat_groups_at_rank_i = self.flat_groups[rank_i]
+            start_group_id = None
+            end_group_id = None
+            for group_id in range(len(self.flat_groups_offset)):
+                if self.flat_groups_offset[group_id] <= self.offset < self.flat_groups_offset[group_id + 1]:
+                    start_group_id = group_id
+                if self.flat_groups_offset[group_id] < end_idx <= self.flat_groups_offset[group_id + 1]:
+                    end_group_id = group_id
+                    break
+            # collect weights from related group/groups
+            for group_id in range(start_group_id, end_group_id + 1):
+                flat_tensor = flat_groups_at_rank_i[group_id]
+                start_offset = self.offset - self.flat_groups_offset[group_id]
+                end_offset = min(end_idx, self.flat_groups_offset[group_id + 1]) - self.flat_groups_offset[group_id]
+                pad_flat_param_chunks.append(flat_tensor[start_offset:end_offset])
+
+        # collect weights from all ranks
+        pad_flat_param = torch.cat(pad_flat_param_chunks, dim=0)
+        param = pad_flat_param[:self.shape.numel()].view(self.shape).contiguous()
+        return param
+
+
+def _zero3_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states):
+    param_shapes = zero_model_states[0].param_shapes
+    avail_numel = sum([flat_group.numel() for flat_group in fp32_flat_groups[0]]) * world_size
+
+    # Reconstruction protocol: For zero3 we need to zip the partitions together at boundary of each
+    # param, re-consolidating each param, while dealing with padding if any
+
+    # merge list of dicts, preserving order
+    param_shapes = {k: v for d in param_shapes for k, v in d.items()}
+
+    if debug:
+        for i in range(world_size):
+            print(f"{FP32_FLAT_GROUPS}[{i}].shape={fp32_flat_groups[i].shape}")
+
+        wanted_params = len(param_shapes)
+        wanted_numel = sum(shape.numel() for shape in param_shapes.values())
+        # not asserting if there is a mismatch due to possible padding
+        avail_numel = fp32_flat_groups[0].numel() * world_size
+        print(f"Trainable params: Have {avail_numel} numels to process.")
+        print(f"Trainable params: Need {wanted_numel} numels in {wanted_params} params.")
+
+    # params
+    # XXX: for huge models that can't fit into the host's RAM we will have to recode this to support
+    # out-of-core computing solution
+    offset = 0
+    total_numel = 0
+    total_params = 0
+    flat_groups_offset = [0] + list(np.cumsum([flat_tensor.numel() for flat_tensor in fp32_flat_groups[0]]))
+    for name, shape in tqdm(param_shapes.items(), desc='Gathering sharded weights'):
+        unpartitioned_numel = shape.numel()
+        total_numel += unpartitioned_numel
+        total_params += 1
+        partitioned_numel, partitioned_padding_numel = zero3_partitioned_param_info(unpartitioned_numel, world_size)
+
+        if debug:
+            print(
+                f"Trainable params: {total_params} {name} full shape: {shape} partition0 numel={partitioned_numel} partitioned_padding_numel={partitioned_padding_numel}"
+            )
+
+        # memory efficient tensor
+        tensor = GatheredTensor(fp32_flat_groups, flat_groups_offset, offset, partitioned_numel, shape)
+        state_dict[name] = tensor
+        offset += partitioned_numel
+
+    offset *= world_size
+
+    # Sanity check
+    if offset != avail_numel:
+        raise ValueError(f"consumed {offset} numels out of {avail_numel} - something is wrong")
+
+    print(f"Reconstructed Trainable fp32 state dict with {total_params} params {total_numel} elements")
+
+
+def _get_fp32_state_dict_from_zero3_checkpoint(world_size, fp32_flat_groups, zero_model_states,
+                                               exclude_frozen_parameters):
+    state_dict = OrderedDict()
+
+    # buffers
+    buffers = zero_model_states[0].buffers
+    state_dict.update(buffers)
+    if debug:
+        print(f"added {len(buffers)} buffers")
+
+    if not exclude_frozen_parameters:
+        _zero3_merge_frozen_params(state_dict, world_size, zero_model_states)
+
+    _zero3_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states)
+
+    # recover shared parameters
+    for pair in zero_model_states[0].shared_params:
+        if pair[1] in state_dict:
+            state_dict[pair[0]] = state_dict[pair[1]]
+
+    return state_dict
+
+
+def to_torch_tensor(state_dict, return_empty_tensor=False):
+    """
+    Convert state_dict of GatheredTensor to torch tensor
+    """
+    torch_state_dict = {}
+    converted_tensors = {}
+    for name, tensor in state_dict.items():
+        tensor_id = id(tensor)
+        if tensor_id in converted_tensors:  # shared tensors
+            shared_tensor = torch_state_dict[converted_tensors[tensor_id]]
+            torch_state_dict[name] = shared_tensor
+        else:
+            converted_tensors[tensor_id] = name
+            if return_empty_tensor:
+                torch_state_dict[name] = torch.empty(tensor.shape, dtype=tensor.dtype)
+            else:
+                torch_state_dict[name] = tensor.contiguous()
+    return torch_state_dict
+
+
+def get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir,
+                                             tag=None,
+                                             exclude_frozen_parameters=False,
+                                             lazy_mode=False):
+    """
+    Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated state_dict that can be loaded with
+    ``load_state_dict()`` and used for training without DeepSpeed or shared with others, for example
+    via a model hub.
+
+    Args:
+        - ``checkpoint_dir``: path to the desired checkpoint folder
+        - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in 'latest' file. e.g., ``global_step14``
+        - ``exclude_frozen_parameters``: exclude frozen parameters
+        - ``lazy_mode``: get state_dict in lazy mode. It returns a dict of pesduo tensor instead of torch tensor, which is more memory efficient.
+          Convert the pesduo tensor to torch tensor by ``.contiguous()``
+
+    Returns:
+        - pytorch ``state_dict``
+
+    A typical usage might be ::
+
+        from deepspeed.utils.zero_to_fp32 import get_fp32_state_dict_from_zero_checkpoint
+        # do the training and checkpoint saving
+        state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir) # already on cpu
+        model = model.cpu() # move to cpu
+        model.load_state_dict(state_dict)
+        # submit to model hub or save the model to share with others
+
+    In this example the ``model`` will no longer be usable in the deepspeed context of the same
+    application. i.e. you will need to re-initialize the deepspeed engine, since
+    ``model.load_state_dict(state_dict)`` will remove all the deepspeed magic from it.
+
+    If you want it all done for you, use ``load_state_dict_from_zero_checkpoint`` instead.
+
+    Note: the above usage may not work if your application doesn't have sufficient free CPU memory.
+    You may need to use the offline approach using the ``zero_to_fp32.py`` script that is saved with
+    the checkpoint. Or you can load state_dict in lazy mode ::
+
+        from deepspeed.utils.zero_to_fp32 import get_fp32_state_dict_from_zero_checkpoint
+        state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir, lazy_mode=True) # not on cpu
+        for name, lazy_tensor in state_dict.item():
+            tensor = lazy_tensor.contiguous()  # to cpu
+            print(name, tensor)
+            # del tensor to release memory if it no longer in use
+    """
+    if tag is None:
+        latest_path = os.path.join(checkpoint_dir, 'latest')
+        if os.path.isfile(latest_path):
+            with open(latest_path, 'r') as fd:
+                tag = fd.read().strip()
+        else:
+            raise ValueError(f"Unable to find 'latest' file at {latest_path}")
+
+    ds_checkpoint_dir = os.path.join(checkpoint_dir, tag)
+
+    if not os.path.isdir(ds_checkpoint_dir):
+        raise FileNotFoundError(f"Directory '{ds_checkpoint_dir}' doesn't exist")
+
+    state_dict = _get_fp32_state_dict_from_zero_checkpoint(ds_checkpoint_dir, exclude_frozen_parameters)
+    if lazy_mode:
+        return state_dict
+    else:
+        return to_torch_tensor(state_dict)
+
+
+def convert_zero_checkpoint_to_fp32_state_dict(checkpoint_dir,
+                                               output_dir,
+                                               max_shard_size="5GB",
+                                               safe_serialization=False,
+                                               tag=None,
+                                               exclude_frozen_parameters=False):
+    """
+    Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated ``state_dict`` file that can be
+    loaded with ``torch.load(file)`` + ``load_state_dict()`` and used for training without DeepSpeed.
+
+    Args:
+        - ``checkpoint_dir``: path to the desired checkpoint folder. (one that contains the tag-folder, like ``global_step14``)
+        - ``output_dir``: directory to the pytorch fp32 state_dict output files
+        - ``max_shard_size``: the maximum size for a checkpoint before being sharded, default value is 5GB
+        - ``safe_serialization``:  whether to save the model using `safetensors` or the traditional PyTorch way (that uses `pickle`).
+        - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in the file named ``latest`` in the checkpoint folder, e.g., ``global_step14``
+        - ``exclude_frozen_parameters``: exclude frozen parameters
+    """
+
+    # Dependency pre-check
+    if safe_serialization:
+        try:
+            from safetensors.torch import save_file
+        except ImportError:
+            print('If you want to use `safe_serialization`, please `pip install safetensors`')
+            raise
+    if max_shard_size is not None:
+        try:
+            from huggingface_hub import split_torch_state_dict_into_shards
+        except ImportError:
+            print('If you want to use `max_shard_size`, please `pip install huggingface_hub`')
+            raise
+
+    # Convert zero checkpoint to state_dict
+    state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir,
+                                                          tag,
+                                                          exclude_frozen_parameters,
+                                                          lazy_mode=True)
+
+    # Shard the model if it is too big.
+    weights_name = "model.safetensors" if safe_serialization else "pytorch_model.bin"
+    if max_shard_size is not None:
+        filename_pattern = weights_name.replace(".bin", "{suffix}.bin").replace(".safetensors", "{suffix}.safetensors")
+        # an memory-efficient approach for sharding
+        empty_state_dict = to_torch_tensor(state_dict, return_empty_tensor=True)
+        state_dict_split = split_torch_state_dict_into_shards(empty_state_dict,
+                                                              filename_pattern=filename_pattern,
+                                                              max_shard_size=max_shard_size)
+    else:
+        from collections import namedtuple
+        StateDictSplit = namedtuple("StateDictSplit", ["is_sharded", "filename_to_tensors"])
+        state_dict_split = StateDictSplit(is_sharded=False,
+                                          filename_to_tensors={weights_name: list(state_dict.keys())})
+
+    # Save the model by shard
+    os.makedirs(output_dir, exist_ok=True)
+    filename_to_tensors = state_dict_split.filename_to_tensors.items()
+    for shard_file, tensors in tqdm(filename_to_tensors, desc="Saving checkpoint shards"):
+        shard_state_dict = {tensor_name: state_dict[tensor_name] for tensor_name in tensors}
+        shard_state_dict = to_torch_tensor(shard_state_dict)
+        output_path = os.path.join(output_dir, shard_file)
+        if safe_serialization:
+            save_file(shard_state_dict, output_path, metadata={"format": "pt"})
+        else:
+            torch.save(shard_state_dict, output_path)
+        # release the memory of current shard
+        for tensor_name in list(shard_state_dict.keys()):
+            del state_dict[tensor_name]
+            del shard_state_dict[tensor_name]
+        del shard_state_dict
+        gc.collect()
+
+    # Save index if sharded
+    if state_dict_split.is_sharded:
+        index = {
+            "metadata": state_dict_split.metadata,
+            "weight_map": state_dict_split.tensor_to_filename,
+        }
+        save_index_file = "model.safetensors.index.json" if safe_serialization else "pytorch_model.bin.index.json"
+        save_index_file = os.path.join(output_dir, save_index_file)
+        with open(save_index_file, "w", encoding="utf-8") as f:
+            content = json.dumps(index, indent=2, sort_keys=True) + "\n"
+            f.write(content)
+
+
+def load_state_dict_from_zero_checkpoint(model, checkpoint_dir, tag=None):
+    """
+    1. Put the provided model to cpu
+    2. Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated ``state_dict``
+    3. Load it into the provided model
+
+    Args:
+        - ``model``: the model object to update
+        - ``checkpoint_dir``: path to the desired checkpoint folder. (one that contains the tag-folder, like ``global_step14``)
+        - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in the file named ``latest`` in the checkpoint folder, e.g., ``global_step14``
+
+    Returns:
+        - ``model`: modified model
+
+    Make sure you have plenty of CPU memory available before you call this function. If you don't
+    have enough use the ``zero_to_fp32.py`` utility to do the conversion. You will find it
+    conveniently placed for you in the checkpoint folder.
+
+    A typical usage might be ::
+
+        from deepspeed.utils.zero_to_fp32 import load_state_dict_from_zero_checkpoint
+        model = load_state_dict_from_zero_checkpoint(trainer.model, checkpoint_dir)
+        # submit to model hub or save the model to share with others
+
+    Note, that once this was run, the ``model`` will no longer be usable in the deepspeed context
+    of the same application. i.e. you will need to re-initialize the deepspeed engine, since
+    ``model.load_state_dict(state_dict)`` will remove all the deepspeed magic from it.
+
+    """
+    logger.info(f"Extracting fp32 weights")
+    state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir, tag)
+
+    logger.info(f"Overwriting model with fp32 weights")
+    model = model.cpu()
+    model.load_state_dict(state_dict, strict=False)
+
+    return model
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("checkpoint_dir",
+                        type=str,
+                        help="path to the desired checkpoint folder, e.g., path/checkpoint-12")
+    parser.add_argument("output_dir",
+                        type=str,
+                        help="directory to the pytorch fp32 state_dict output files"
+                        "(e.g. path/checkpoint-12-output/)")
+    parser.add_argument(
+        "--max_shard_size",
+        type=str,
+        default="5GB",
+        help="The maximum size for a checkpoint before being sharded. Checkpoints shard will then be each of size"
+        "lower than this size. If expressed as a string, needs to be digits followed by a unit (like `5MB`"
+        "We default it to 5GB in order for models to be able to run easily on free-tier google colab instances"
+        "without CPU OOM issues.")
+    parser.add_argument(
+        "--safe_serialization",
+        default=False,
+        action='store_true',
+        help="Whether to save the model using `safetensors` or the traditional PyTorch way (that uses `pickle`).")
+    parser.add_argument("-t",
+                        "--tag",
+                        type=str,
+                        default=None,
+                        help="checkpoint tag used as a unique identifier for checkpoint. e.g., global_step1")
+    parser.add_argument("--exclude_frozen_parameters", action='store_true', help="exclude frozen parameters")
+    parser.add_argument("-d", "--debug", action='store_true', help="enable debug")
+    args = parser.parse_args()
+
+    debug = args.debug
+
+    convert_zero_checkpoint_to_fp32_state_dict(args.checkpoint_dir,
+                                               args.output_dir,
+                                               max_shard_size=args.max_shard_size,
+                                               safe_serialization=args.safe_serialization,
+                                               tag=args.tag,
+                                               exclude_frozen_parameters=args.exclude_frozen_parameters)
diff --git a/en/baseline/data_15000_1000/eval_results.json b/en/baseline/data_15000_1000/eval_results.json
new file mode 100644
index 0000000000000000000000000000000000000000..3fb225fed1ba93f08bf75d1e83000a5774b4a09a
--- /dev/null
+++ b/en/baseline/data_15000_1000/eval_results.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:998d8373832fac0bf150b5805f2ce7b2abbfa920abd95421ac12160ecfab50d0
+size 267
diff --git a/en/baseline/data_15000_1000/special_tokens_map.json b/en/baseline/data_15000_1000/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..4ed2bd8240878a7a0d4fd2c60cdc89f6d7a5f1e1
--- /dev/null
+++ b/en/baseline/data_15000_1000/special_tokens_map.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83da1082df286d75a2984dc06ec439f4febc3d862ac55d199402e5d345f5773a
+size 372
diff --git a/en/baseline/data_15000_1000/tokenizer.json b/en/baseline/data_15000_1000/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..66cd9d7e0daec95eb10d16a63c615637dbbb7304
--- /dev/null
+++ b/en/baseline/data_15000_1000/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79e3e522635f3171300913bb421464a87de6222182a0570b9b2ccba2a964b2b4
+size 9085657
diff --git a/en/baseline/data_15000_1000/tokenizer_config.json b/en/baseline/data_15000_1000/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..7144ada11807e90b92529f17434f8d01915c3dff
--- /dev/null
+++ b/en/baseline/data_15000_1000/tokenizer_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d967a51bb800d3e471ea23dd3f7b054b136600238bbbb40612b8b96b0370746e
+size 55427
diff --git a/en/baseline/data_15000_1000/train.log b/en/baseline/data_15000_1000/train.log
new file mode 100644
index 0000000000000000000000000000000000000000..40e54eaa08e434aaa52db7ac314687ca35d4ec7b
--- /dev/null
+++ b/en/baseline/data_15000_1000/train.log
@@ -0,0 +1,3319 @@
+W0626 22:11:20.302608 1353303 site-packages/torch/distributed/run.py:766] 
+W0626 22:11:20.302608 1353303 site-packages/torch/distributed/run.py:766] *****************************************
+W0626 22:11:20.302608 1353303 site-packages/torch/distributed/run.py:766] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+W0626 22:11:20.302608 1353303 site-packages/torch/distributed/run.py:766] *****************************************
+[2025-06-26 22:11:26,178] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-26 22:11:26,228] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-26 22:11:26,853] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-26 22:11:26,947] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-26 22:11:27,682] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-26 22:11:27,807] [INFO] [comm.py:675:init_distributed] cdb=None
+[2025-06-26 22:11:27,837] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-26 22:11:27,962] [INFO] [comm.py:675:init_distributed] cdb=None
+06/26/2025 22:11:28 - WARNING - __main__ - Process rank: 2, device: cuda:2, n_gpu: 1distributed training: True, 16-bits training: False
+06/26/2025 22:11:28 - WARNING - __main__ - Process rank: 1, device: cuda:1, n_gpu: 1distributed training: True, 16-bits training: False
+[2025-06-26 22:11:28,454] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-26 22:11:28,474] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-26 22:11:28,579] [INFO] [comm.py:675:init_distributed] cdb=None
+[2025-06-26 22:11:28,612] [INFO] [comm.py:675:init_distributed] cdb=None
+[2025-06-26 22:11:28,612] [INFO] [comm.py:706:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
+06/26/2025 22:11:28 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1distributed training: True, 16-bits training: False
+06/26/2025 22:11:28 - INFO - __main__ - Training/evaluation parameters LoRATrainingArguments(
+_n_gpu=1,
+accelerator_config={'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None, 'use_configured_state': False},
+adafactor=False,
+adam_beta1=0.9,
+adam_beta2=0.999,
+adam_epsilon=1e-08,
+auto_find_batch_size=False,
+batch_eval_metrics=False,
+bf16=True,
+bf16_full_eval=True,
+data_seed=None,
+dataloader_drop_last=False,
+dataloader_num_workers=2,
+dataloader_persistent_workers=False,
+dataloader_pin_memory=True,
+dataloader_prefetch_factor=None,
+ddp_backend=None,
+ddp_broadcast_buffers=None,
+ddp_bucket_cap_mb=None,
+ddp_find_unused_parameters=None,
+ddp_timeout=3600,
+debug=[],
+deepspeed=./config/deepspeed_config.json,
+disable_tqdm=False,
+dispatch_batches=None,
+do_eval=True,
+do_predict=False,
+do_train=True,
+eval_accumulation_steps=None,
+eval_delay=0,
+eval_do_concat_batches=True,
+eval_on_start=True,
+eval_steps=200,
+eval_strategy=steps,
+eval_use_gather_object=False,
+evaluation_strategy=None,
+fp16=False,
+fp16_backend=auto,
+fp16_full_eval=False,
+fp16_opt_level=O1,
+fsdp=[],
+fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False},
+fsdp_min_num_params=0,
+fsdp_transformer_layer_cls_to_wrap=None,
+full_determinism=False,
+gradient_accumulation_steps=1,
+gradient_checkpointing=True,
+gradient_checkpointing_kwargs=None,
+greater_is_better=False,
+group_by_length=False,
+half_precision_backend=auto,
+hub_always_push=False,
+hub_model_id=None,
+hub_private_repo=False,
+hub_strategy=every_save,
+hub_token=<HUB_TOKEN>,
+ignore_data_skip=False,
+include_inputs_for_metrics=False,
+include_num_input_tokens_seen=False,
+include_tokens_per_second=False,
+jit_mode_eval=False,
+label_names=None,
+label_smoothing_factor=0.0,
+learning_rate=0.0005,
+length_column_name=length,
+load_best_model_at_end=True,
+load_lora_from=None,
+local_rank=0,
+log_level=passive,
+log_level_replica=warning,
+log_on_each_node=True,
+logging_dir=./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/runs/Jun26_22-11-26_innmi1srh2-p040,
+logging_first_step=False,
+logging_nan_inf_filter=True,
+logging_steps=1.0,
+logging_strategy=steps,
+lora_config=./config/lora_config.json,
+lr_scheduler_kwargs={},
+lr_scheduler_type=inverse_sqrt,
+max_grad_norm=1.0,
+max_steps=-1,
+metric_for_best_model=eval_loss,
+mp_parameters=,
+neftune_noise_alpha=None,
+no_cuda=False,
+num_train_epochs=5.0,
+optim=adamw_torch,
+optim_args=None,
+optim_target_modules=None,
+output_dir=./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/,
+overwrite_output_dir=True,
+past_index=-1,
+per_device_eval_batch_size=25,
+per_device_train_batch_size=25,
+prediction_loss_only=False,
+push_to_hub=False,
+push_to_hub_model_id=None,
+push_to_hub_organization=None,
+push_to_hub_token=<PUSH_TO_HUB_TOKEN>,
+ray_scope=last,
+remove_unused_columns=True,
+report_to=['wandb'],
+restore_callback_states_from_checkpoint=False,
+resume_from_checkpoint=None,
+run_name=./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/,
+save_on_each_node=False,
+save_only_model=False,
+save_safetensors=True,
+save_steps=200,
+save_strategy=steps,
+save_total_limit=1,
+seed=1,
+skip_memory_metrics=True,
+split_batches=None,
+tf32=None,
+torch_compile=False,
+torch_compile_backend=None,
+torch_compile_mode=None,
+torch_empty_cache_steps=None,
+torchdynamo=None,
+tpu_metrics_debug=False,
+tpu_num_cores=None,
+use_cpu=False,
+use_int8_training=False,
+use_ipex=False,
+use_legacy_prediction_loop=False,
+use_lora=True,
+use_mps_device=False,
+warmup_ratio=0.03,
+warmup_steps=0,
+weight_decay=0.0,
+)
+06/26/2025 22:11:28 - WARNING - __main__ - Process rank: 3, device: cuda:3, n_gpu: 1distributed training: True, 16-bits training: False
+Using custom data configuration default-0b5518de39c6fcc5
+06/26/2025 22:11:29 - INFO - datasets.builder - Using custom data configuration default-0b5518de39c6fcc5
+Loading Dataset Infos from /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/datasets/packaged_modules/json
+06/26/2025 22:11:29 - INFO - datasets.info - Loading Dataset Infos from /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/datasets/packaged_modules/json
+Overwrite dataset info from restored data version if exists.
+06/26/2025 22:11:29 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists.
+Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+06/26/2025 22:11:29 - INFO - datasets.info - Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+Found cached dataset json (/home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092)
+06/26/2025 22:11:29 - INFO - datasets.builder - Found cached dataset json (/home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092)
+Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+06/26/2025 22:11:29 - INFO - datasets.info - Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+[WARNING|logging.py:328] 2025-06-26 22:11:29,621 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s][WARNING|logging.py:328] 2025-06-26 22:11:29,837 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+[INFO|configuration_utils.py:733] 2025-06-26 22:11:29,856 >> loading configuration file config.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/config.json
+[INFO|configuration_utils.py:821] 2025-06-26 22:11:29,859 >> Model config LlamaConfig {
+  "_name_or_path": "meta-llama/Llama-3.1-8B-Instruct",
+  "additional_loss_layer": 16,
+  "alignment_matrices_path": null,
+  "apply_inverse": false,
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "contrastive_loss_temperature": 1.0,
+  "contrastive_loss_weight": 1.0,
+  "contrastive_pooling_type": "mean",
+  "distance_function": "cosine",
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "inject_Ws": false,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "only_train_contrastive": false,
+  "only_train_language_modeling": true,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.44.0.dev0",
+  "unidirectional_contrastive_loss": false,
+  "use_cache": true,
+  "vocab_size": 128256
+}
+
+
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  4.29it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  5.06it/s][INFO|tokenization_utils_base.py:2269] 2025-06-26 22:11:30,121 >> loading file tokenizer.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/tokenizer.json
+[INFO|tokenization_utils_base.py:2269] 2025-06-26 22:11:30,121 >> loading file added_tokens.json from cache at None
+[INFO|tokenization_utils_base.py:2269] 2025-06-26 22:11:30,121 >> loading file special_tokens_map.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/special_tokens_map.json
+[INFO|tokenization_utils_base.py:2269] 2025-06-26 22:11:30,121 >> loading file tokenizer_config.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/tokenizer_config.json
+
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  3.84it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  5.22it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  4.43it/s][WARNING|logging.py:328] 2025-06-26 22:11:30,436 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+[INFO|tokenization_utils_base.py:2513] 2025-06-26 22:11:30,438 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
+06/26/2025 22:11:30 - INFO - __main__ - Tokenizer is fast: True
+[INFO|modeling_utils.py:3667] 2025-06-26 22:11:30,441 >> loading weights file model.safetensors from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/model.safetensors.index.json
+[INFO|modeling_utils.py:1591] 2025-06-26 22:11:30,442 >> Instantiating LlamaForCausalLM model under default dtype torch.bfloat16.
+[WARNING|logging.py:328] 2025-06-26 22:11:30,444 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+[INFO|configuration_utils.py:1038] 2025-06-26 22:11:30,445 >> Generate config GenerationConfig {
+  "bos_token_id": 128000,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ]
+}
+
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.52it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.29it/s]
+
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  5.00it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.82it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.26it/s]
+adding special tokens...
+
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  4.68it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  4.24it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  5.84it/s]adding special tokens...
+
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  5.49it/s]trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  6.46it/s]PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank2]:[W626 22:11:31.743908953 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 2]  using GPU 2 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  5.90it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  7.03it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  6.52it/s]
+[INFO|modeling_utils.py:4499] 2025-06-26 22:11:31,135 >> All model checkpoint weights were used when initializing LlamaForCausalLM.
+
+[INFO|modeling_utils.py:4507] 2025-06-26 22:11:31,135 >> All the weights of LlamaForCausalLM were initialized from the model checkpoint at meta-llama/Llama-3.1-8B-Instruct.
+If your task is similar to the task the model of the checkpoint was trained on, you can already use LlamaForCausalLM for predictions without further training.
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  6.40it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.96it/s]
+trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank1]:[W626 22:11:31.974470572 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 1]  using GPU 1 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+[INFO|configuration_utils.py:993] 2025-06-26 22:11:31,375 >> loading configuration file generation_config.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/generation_config.json
+[INFO|configuration_utils.py:1038] 2025-06-26 22:11:31,376 >> Generate config GenerationConfig {
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "temperature": 0.6,
+  "top_p": 0.9
+}
+
+adding special tokens...
+06/26/2025 22:11:31 - INFO - __main__ - ================ pad, eos, bos, unk, padding ================ 
+06/26/2025 22:11:31 - INFO - __main__ - <|eot_id|>, 128009
+06/26/2025 22:11:31 - INFO - __main__ - <|eot_id|>, 128009
+06/26/2025 22:11:31 - INFO - __main__ - <|begin_of_text|>, 128000
+06/26/2025 22:11:31 - INFO - __main__ - <|reserved_special_token_0|>, 128002
+06/26/2025 22:11:31 - INFO - __main__ - right
+06/26/2025 22:11:31 - INFO - __main__ - lora_r : 8
+06/26/2025 22:11:31 - INFO - __main__ - lora_alpha : 16
+06/26/2025 22:11:31 - INFO - __main__ - lora_dropout : 0.1
+06/26/2025 22:11:31 - INFO - __main__ - lora_target_modules : ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj']
+06/26/2025 22:11:31 - INFO - __main__ - LoRA configs: LoraConfig(task_type='CAUSAL_LM', peft_type=<PeftType.LORA: 'LORA'>, auto_mapping=None, base_model_name_or_path=None, revision=None, inference_mode=False, r=8, target_modules={'o_proj', 'down_proj', 'up_proj', 'k_proj', 'v_proj', 'q_proj', 'gate_proj'}, exclude_modules=None, lora_alpha=16, lora_dropout=0.1, fan_in_fan_out=False, bias='none', use_rslora=False, modules_to_save=None, init_lora_weights=True, layers_to_transform=None, layers_pattern=None, rank_pattern={}, alpha_pattern={}, megatron_config=None, megatron_core='megatron.core', trainable_token_indices=None, loftq_config={}, eva_config=None, corda_config=None, use_dora=False, layer_replication=None, runtime_config=LoraRuntimeConfig(ephemeral_gpu_offload=False), lora_bias=False)
+adding special tokens...
+trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+06/26/2025 22:11:31 - INFO - __main__ - block size: 2048
+trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank3]:[W626 22:11:31.487471354 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 3]  using GPU 3 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-509de0467929df64.arrow
+06/26/2025 22:11:31 - INFO - datasets.arrow_dataset - Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-509de0467929df64.arrow
+Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-61aa8bb2d831e11e.arrow
+06/26/2025 22:11:31 - INFO - datasets.arrow_dataset - Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-61aa8bb2d831e11e.arrow
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank0]:[W626 22:11:32.786162814 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 0]  using GPU 0 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+06/26/2025 22:11:33 - INFO - __main__ - xxx: Showcase the tokenized training samples.
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+[INFO|trainer.py:658] 2025-06-26 22:11:35,525 >> Using auto half precision backend
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+[2025-06-26 22:11:35,747] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed info: version=0.17.1, git-hash=unknown, git-branch=unknown
+[2025-06-26 22:11:35,747] [INFO] [config.py:655:__init__] Config mesh_device None world_size = 4
+[2025-06-26 22:11:39,348] [INFO] [engine.py:1325:_configure_distributed_model] ********** distributed groups summary **********
+	 self.dp_world_size=4
+	 self.mp_world_size=1
+	 self.seq_dp_world_size=4
+	 self.sequence_parallel_size=1
+***********************************************
+[2025-06-26 22:11:41,378] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Detected CUDA files, patching ldflags
+Emitting ninja build file /home/iitm_admin/.cache/torch_extensions/py39_cu126/cpu_adam/build.ninja...
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/utils/cpp_extension.py:2356: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
+If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
+  warnings.warn(
+Building extension module cpu_adam...
+Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
+ninja: no work to do.
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 2.7634787559509277 seconds
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Detected CUDA files, patching ldflags
+Emitting ninja build file /home/iitm_admin/.cache/torch_extensions/py39_cu126/cpu_adam/build.ninja...
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/utils/cpp_extension.py:2356: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
+If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
+  warnings.warn(
+Building extension module cpu_adam...
+Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
+ninja: no work to do.
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 3.0678343772888184 seconds
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 3.1556124687194824 seconds
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 3.1775965690612793 seconds
+Adam Optimizer #0 is created with AVX512 arithmetic capability.
+Config: alpha=0.000500, betas=(0.900000, 0.999000), weight_decay=0.000000, adam_w=1
+[2025-06-26 22:11:45,941] [INFO] [logging.py:107:log_dist] [Rank 0] Using DeepSpeed Optimizer param name adam as basic optimizer
+[2025-06-26 22:11:45,942] [INFO] [logging.py:107:log_dist] [Rank 0] Removing param_group that has no 'params' in the basic Optimizer
+[2025-06-26 22:11:46,017] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Basic Optimizer = DeepSpeedCPUAdam
+[2025-06-26 22:11:46,017] [INFO] [utils.py:59:is_zero_supported_optimizer] Checking ZeRO support for optimizer=DeepSpeedCPUAdam type=<class 'deepspeed.ops.adam.cpu_adam.DeepSpeedCPUAdam'>
+[2025-06-26 22:11:46,017] [INFO] [logging.py:107:log_dist] [Rank 0] Creating torch.bfloat16 ZeRO stage 1 optimizer
+[2025-06-26 22:11:46,017] [INFO] [stage_1_and_2.py:151:__init__] Reduce bucket size 200000000
+[2025-06-26 22:11:46,017] [INFO] [stage_1_and_2.py:152:__init__] Allgather bucket size 200000000
+[2025-06-26 22:11:46,017] [INFO] [stage_1_and_2.py:153:__init__] CPU Offload: True
+[2025-06-26 22:11:46,017] [INFO] [stage_1_and_2.py:154:__init__] Round robin gradient partitioning: False
+[2025-06-26 22:11:46,397] [INFO] [utils.py:781:see_memory_usage] Before initializing optimizer states
+[2025-06-26 22:11:46,398] [INFO] [utils.py:782:see_memory_usage] MA 15.0 GB         Max_MA 15.0 GB         CA 15.16 GB         Max_CA 15 GB 
+[2025-06-26 22:11:46,398] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 77.31 GB, percent = 3.8%
+[2025-06-26 22:11:46,593] [INFO] [utils.py:781:see_memory_usage] After initializing optimizer states
+[2025-06-26 22:11:46,594] [INFO] [utils.py:782:see_memory_usage] MA 15.0 GB         Max_MA 15.0 GB         CA 15.16 GB         Max_CA 15 GB 
+[2025-06-26 22:11:46,594] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 77.5 GB, percent = 3.8%
+[2025-06-26 22:11:46,594] [INFO] [stage_1_and_2.py:573:__init__] optimizer state initialized
+[2025-06-26 22:11:46,731] [INFO] [utils.py:781:see_memory_usage] After initializing ZeRO optimizer
+[2025-06-26 22:11:46,731] [INFO] [utils.py:782:see_memory_usage] MA 15.0 GB         Max_MA 15.0 GB         CA 15.16 GB         Max_CA 15 GB 
+[2025-06-26 22:11:46,732] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 77.55 GB, percent = 3.8%
+[2025-06-26 22:11:46,734] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Final Optimizer = DeepSpeedZeroOptimizer
+[2025-06-26 22:11:46,734] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed using client callable to create LR scheduler
+[2025-06-26 22:11:46,735] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed LR Scheduler = <torch.optim.lr_scheduler.LambdaLR object at 0x73a66c3e5490>
+[2025-06-26 22:11:46,735] [INFO] [logging.py:107:log_dist] [Rank 0] step=0, skipped=0, lr=[0.0], mom=[[0.9, 0.999]]
+[2025-06-26 22:11:46,740] [INFO] [logging.py:107:log_dist] [Rank 0] [TorchCheckpointEngine] Initialized with serialization = True
+[2025-06-26 22:11:46,740] [INFO] [config.py:921:print] DeepSpeedEngine configuration:
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   activation_checkpointing_config  {
+    "partition_activations": false, 
+    "contiguous_memory_optimization": false, 
+    "cpu_checkpointing": false, 
+    "number_checkpoints": null, 
+    "synchronize_checkpoint_boundary": false, 
+    "profile": false
+}
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'intra_op_parallelism': 1, 'single_submit': False, 'overlap_events': True, 'use_gds': False}
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   amp_enabled .................. False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   amp_params ................... False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   autotuning_config ............ {
+    "enabled": false, 
+    "start_step": null, 
+    "end_step": null, 
+    "metric_path": null, 
+    "arg_mappings": null, 
+    "metric": "throughput", 
+    "model_info": null, 
+    "results_dir": "autotuning_results", 
+    "exps_dir": "autotuning_exps", 
+    "overwrite": true, 
+    "fast": true, 
+    "start_profile_step": 3, 
+    "end_profile_step": 5, 
+    "tuner_type": "gridsearch", 
+    "tuner_early_stopping": 5, 
+    "tuner_num_trials": 50, 
+    "model_info_path": null, 
+    "mp_size": 1, 
+    "max_train_batch_size": null, 
+    "min_train_batch_size": 1, 
+    "max_train_micro_batch_size_per_gpu": 1.024000e+03, 
+    "min_train_micro_batch_size_per_gpu": 1, 
+    "num_tuning_micro_batch_sizes": 3
+}
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   bfloat16_config .............. enabled=True immediate_grad_update=False check_grad_overflow=False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   checkpoint_config ............ {'tag_validation': 'WARN', 'checkpoint_serialization': True, 'writer': None}
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   checkpoint_parallel_write_pipeline  False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   checkpoint_tag_validation_enabled  True
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   checkpoint_tag_validation_fail  False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   comms_config ................. <deepspeed.comm.config.DeepSpeedCommsConfig object at 0x73a66d183b20>
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   communication_data_type ...... None
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   compile_config ............... deepcompile=False free_activation=False offload_activation=False offload_opt_states=False double_buffer=True symmetric_memory=False debug_log=False offload_parameters=False sync_before_reduce=False sync_after_reduce=False sync_before_allgather=False sync_after_allgather=False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}}
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   curriculum_enabled_legacy .... False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   curriculum_params_legacy ..... False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'pin_memory': False, 'curriculum_learning': {'enabled': False}, 'dynamic_batching': {'enabled': False, 'lr_scaling_method': 'linear', 'min_batch_size': 1, 'max_batch_size': None, 'sequence_picking_order': 'dataloader', 'verbose': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}}
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   data_efficiency_enabled ...... False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   dataloader_drop_last ......... False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   disable_allgather ............ False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   dump_state ................... False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   eigenvalue_enabled ........... False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   eigenvalue_gas_boundary_resolution  1
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   eigenvalue_layer_name ........ bert.encoder.layer
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   eigenvalue_layer_num ......... 0
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   eigenvalue_max_iter .......... 100
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   eigenvalue_stability ......... 1e-06
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   eigenvalue_tol ............... 0.01
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   eigenvalue_verbose ........... False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   elasticity_enabled ........... False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   float16_config ............... enabled=False auto_cast=False loss_scale=0.0 initial_scale_power=16 loss_scale_window=1000 hysteresis=2 consecutive_hysteresis=False min_loss_scale=1 fp16_master_weights_and_grads=False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   flops_profiler_config ........ {
+    "enabled": false, 
+    "recompute_fwd_factor": 0.0, 
+    "profile_step": 1, 
+    "module_depth": -1, 
+    "top_modules": 1, 
+    "detailed": true, 
+    "output_file": null
+}
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   global_rank .................. 0
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   grad_accum_dtype ............. None
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   gradient_accumulation_steps .. 1
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   gradient_clipping ............ 1.0
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   gradient_predivide_factor .... 1.0
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   graph_harvesting ............. False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   load_universal_checkpoint .... False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   memory_breakdown ............. False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   mics_hierarchial_params_gather  False
+[2025-06-26 22:11:46,741] [INFO] [config.py:925:print]   mics_shard_size .............. -1
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') comet=CometConfig(enabled=False, samples_log_interval=100, project=None, workspace=None, api_key=None, experiment_name=None, experiment_key=None, online=None, mode=None) wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName')
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   nebula_config ................ {
+    "enabled": false, 
+    "persistent_storage_path": null, 
+    "persistent_time_interval": 100, 
+    "num_of_version_in_retention": 2, 
+    "enable_nebula_load": true, 
+    "load_path": null
+}
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   optimizer_legacy_fusion ...... False
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   optimizer_name ............... adam
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   optimizer_params ............. {'lr': 0.0005, 'betas': [0.9, 0.999], 'eps': 1e-08, 'weight_decay': 0.0}
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0, 'pipe_partitioned': True, 'grad_partitioned': True}
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   pld_enabled .................. False
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   pld_params ................... False
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   prescale_gradients ........... False
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   scheduler_name ............... None
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   scheduler_params ............. None
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   seq_parallel_communication_data_type  torch.float32
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   sparse_attention ............. None
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   sparse_gradients_enabled ..... False
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   steps_per_print .............. inf
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   tensor_parallel_config ....... dtype=torch.float16 autotp_size=0 tp_overlap_comm=False tensor_parallel=TPConfig(tp_size=1, tp_grain_size=1, mpu=None, tp_group=None) injection_policy_tuple=None keep_module_on_host=False replace_with_kernel_inject=False
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   timers_config ................ enabled=True synchronized=True
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   train_batch_size ............. 100
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   train_micro_batch_size_per_gpu  25
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   use_data_before_expert_parallel_  False
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   use_node_local_storage ....... False
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   wall_clock_breakdown ......... False
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   weight_quantization_config ... None
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   world_size ................... 4
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   zero_allow_untested_optimizer  False
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   zero_config .................. stage=1 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=200000000 use_multi_rank_bucket_allreduce=True allgather_partitions=True allgather_bucket_size=200000000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=None offload_optimizer=DeepSpeedZeroOffloadOptimizerConfig(device='cpu', nvme_path=None, buffer_count=4, pin_memory=True, pipeline_read=False, pipeline_write=False, fast_init=False, ratio=1.0) sub_group_size=1000000000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50000000 param_persistence_threshold=100000 model_persistence_threshold=9223372036854775807 max_live_parameters=1000000000 max_reuse_distance=1000000000 gather_16bit_weights_on_model_save=False module_granularity_threshold=0 use_all_reduce_for_fetch_params=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False zero_hpz_partition_size=1 zero_quantized_weights=False zero_quantized_nontrainable_weights=False zero_quantized_gradients=False zeropp_loco_param=None mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=True pipeline_loading_checkpoint=False override_module_apply=True log_trace_cache_warnings=False
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   zero_enabled ................. True
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   zero_force_ds_cpu_optimizer .. True
+[2025-06-26 22:11:46,742] [INFO] [config.py:925:print]   zero_optimization_stage ...... 1
+[2025-06-26 22:11:46,742] [INFO] [config.py:911:print_user_config]   json = {
+    "optimizer": {
+        "type": "Adam", 
+        "params": {
+            "lr": 0.0005, 
+            "betas": [0.9, 0.999], 
+            "eps": 1e-08, 
+            "weight_decay": 0.0
+        }
+    }, 
+    "bf16": {
+        "enabled": true
+    }, 
+    "fp16": {
+        "enabled": false, 
+        "loss_scale": 0, 
+        "loss_scale_window": 1000, 
+        "initial_scale_power": 16, 
+        "hysteresis": 2, 
+        "min_loss_scale": 1
+    }, 
+    "zero_optimization": {
+        "stage": 1, 
+        "offload_optimizer": {
+            "device": "cpu", 
+            "pin_memory": true
+        }, 
+        "allgather_partitions": true, 
+        "allgather_bucket_size": 2.000000e+08, 
+        "overlap_comm": true, 
+        "reduce_scatter": true, 
+        "reduce_bucket_size": 2.000000e+08, 
+        "contiguous_gradients": true
+    }, 
+    "gradient_accumulation_steps": 1, 
+    "gradient_clipping": 1.0, 
+    "steps_per_print": inf, 
+    "train_batch_size": 100, 
+    "train_micro_batch_size_per_gpu": 25, 
+    "wall_clock_breakdown": false
+}
+[INFO|trainer.py:2145] 2025-06-26 22:11:46,744 >> ***** Running training *****
+[INFO|trainer.py:2146] 2025-06-26 22:11:46,744 >>   Num examples = 15,000
+[INFO|trainer.py:2147] 2025-06-26 22:11:46,744 >>   Num Epochs = 5
+[INFO|trainer.py:2148] 2025-06-26 22:11:46,744 >>   Instantaneous batch size per device = 25
+[INFO|trainer.py:2151] 2025-06-26 22:11:46,744 >>   Total train batch size (w. parallel, distributed & accumulation) = 100
+[INFO|trainer.py:2152] 2025-06-26 22:11:46,744 >>   Gradient Accumulation steps = 1
+[INFO|trainer.py:2153] 2025-06-26 22:11:46,744 >>   Total optimization steps = 750
+[INFO|trainer.py:2154] 2025-06-26 22:11:46,747 >>   Number of trainable parameters = 20,971,520
+[INFO|integration_utils.py:807] 2025-06-26 22:11:46,751 >> Automatic Weights & Biases logging enabled, to disable set os.environ["WANDB_DISABLED"] = "true"
+wandb: WARNING The `run_name` is currently set to the same value as `TrainingArguments.output_dir`. If this was not intended, please specify a different run name by setting the `TrainingArguments.run_name` parameter.
+wandb: Currently logged in as: sidharthpulipaka (indic-encoder) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
+wandb: Tracking run with wandb version 0.20.1
+wandb: Run data is saved locally in /home/iitm_admin/llmteam/mid-align/wandb/run-20250626_221147-yiz6vtnu
+wandb: Run `wandb offline` to turn off syncing.
+wandb: Syncing run ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/
+wandb: ⭐️ View project at https://wandb.ai/indic-encoder/midalign
+wandb: 🚀 View run at https://wandb.ai/indic-encoder/midalign/runs/yiz6vtnu
+
  0%|          | 0/750 [00:00<?, ?it/s][INFO|trainer.py:3831] 2025-06-26 22:11:48,802 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-26 22:11:48,803 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-26 22:11:48,803 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:00<00:03,  2.40it/s]
+
 30%|███       | 3/10 [00:01<00:04,  1.74it/s]
+
 40%|████      | 4/10 [00:02<00:03,  1.62it/s]
+
 50%|█████     | 5/10 [00:04<00:05,  1.19s/it]
+
 60%|██████    | 6/10 [00:05<00:04,  1.07s/it]
+
 70%|███████   | 7/10 [00:06<00:02,  1.07it/s]
+
 80%|████████  | 8/10 [00:06<00:01,  1.22it/s]
+
 90%|█████████ | 9/10 [00:07<00:00,  1.19it/s]
+
100%|██████████| 10/10 [00:10<00:00,  1.50s/it]
                                       
+
                                               
{'eval_loss': 0.5909706354141235, 'eval_accuracy': 0.4538808735936466, 'eval_runtime': 23.5755, 'eval_samples_per_second': 42.417, 'eval_steps_per_second': 0.424, 'epoch': 0}
+
  0%|          | 0/750 [00:23<?, ?it/s]
+
100%|██████████| 10/10 [00:14<00:00,  1.50s/it]
+
                                               
  0%|          | 1/750 [00:35<7:19:33, 35.21s/it]
                                                 
{'loss': 0.6291, 'grad_norm': 0.7677975296974182, 'learning_rate': 2.173913043478261e-05, 'epoch': 0.01}
+
  0%|          | 1/750 [00:35<7:19:33, 35.21s/it]
  0%|          | 2/750 [00:37<3:19:02, 15.97s/it]
                                                 
{'loss': 0.5594, 'grad_norm': 0.6497437357902527, 'learning_rate': 4.347826086956522e-05, 'epoch': 0.01}
+
  0%|          | 2/750 [00:37<3:19:02, 15.97s/it]
  0%|          | 3/750 [00:39<1:59:13,  9.58s/it]
                                                 
{'loss': 0.6413, 'grad_norm': 0.888583242893219, 'learning_rate': 6.521739130434782e-05, 'epoch': 0.02}
+
  0%|          | 3/750 [00:39<1:59:13,  9.58s/it]
  1%|          | 4/750 [00:42<1:26:41,  6.97s/it]
                                                 
{'loss': 0.6152, 'grad_norm': 0.8042678236961365, 'learning_rate': 8.695652173913044e-05, 'epoch': 0.03}
+
  1%|          | 4/750 [00:42<1:26:41,  6.97s/it]
  1%|          | 5/750 [00:45<1:08:37,  5.53s/it]
                                                 
{'loss': 0.5155, 'grad_norm': 0.951114296913147, 'learning_rate': 0.00010869565217391305, 'epoch': 0.03}
+
  1%|          | 5/750 [00:45<1:08:37,  5.53s/it]
  1%|          | 6/750 [00:50<1:05:54,  5.32s/it]
                                                 
{'loss': 0.4686, 'grad_norm': 0.6673992276191711, 'learning_rate': 0.00013043478260869564, 'epoch': 0.04}
+
  1%|          | 6/750 [00:50<1:05:54,  5.32s/it]
  1%|          | 7/750 [00:53<58:11,  4.70s/it]  
                                               
{'loss': 0.4482, 'grad_norm': 0.33969762921333313, 'learning_rate': 0.00015217391304347827, 'epoch': 0.05}
+
  1%|          | 7/750 [00:53<58:11,  4.70s/it]
  1%|          | 8/750 [00:56<48:30,  3.92s/it]
                                               
{'loss': 0.4424, 'grad_norm': 0.4537906050682068, 'learning_rate': 0.00017391304347826088, 'epoch': 0.05}
+
  1%|          | 8/750 [00:56<48:30,  3.92s/it]
  1%|          | 9/750 [00:59<44:26,  3.60s/it]
                                               
{'loss': 0.3913, 'grad_norm': 0.3540533483028412, 'learning_rate': 0.0001956521739130435, 'epoch': 0.06}
+
  1%|          | 9/750 [00:59<44:26,  3.60s/it]
  1%|▏         | 10/750 [01:01<40:03,  3.25s/it]
                                                
{'loss': 0.4265, 'grad_norm': 0.36058953404426575, 'learning_rate': 0.0002173913043478261, 'epoch': 0.07}
+
  1%|▏         | 10/750 [01:01<40:03,  3.25s/it]
  1%|▏         | 11/750 [01:04<37:38,  3.06s/it]
                                                
{'loss': 0.3892, 'grad_norm': 0.21993105113506317, 'learning_rate': 0.0002391304347826087, 'epoch': 0.07}
+
  1%|▏         | 11/750 [01:04<37:38,  3.06s/it]
  2%|▏         | 12/750 [01:07<38:40,  3.14s/it]
                                                
{'loss': 0.375, 'grad_norm': 0.1484135240316391, 'learning_rate': 0.0002608695652173913, 'epoch': 0.08}
+
  2%|▏         | 12/750 [01:07<38:40,  3.14s/it]
  2%|▏         | 13/750 [01:10<37:25,  3.05s/it]
                                                
{'loss': 0.4261, 'grad_norm': 0.18072150647640228, 'learning_rate': 0.0002826086956521739, 'epoch': 0.09}
+
  2%|▏         | 13/750 [01:10<37:25,  3.05s/it]
  2%|▏         | 14/750 [01:12<35:51,  2.92s/it]
                                                
{'loss': 0.3101, 'grad_norm': 0.1535647064447403, 'learning_rate': 0.00030434782608695655, 'epoch': 0.09}
+
  2%|▏         | 14/750 [01:13<35:51,  2.92s/it]
  2%|▏         | 15/750 [01:15<33:42,  2.75s/it]
                                                
{'loss': 0.3844, 'grad_norm': 0.19179384410381317, 'learning_rate': 0.00032608695652173916, 'epoch': 0.1}
+
  2%|▏         | 15/750 [01:15<33:42,  2.75s/it]
  2%|▏         | 16/750 [01:18<34:34,  2.83s/it]
                                                
{'loss': 0.3347, 'grad_norm': 0.13737882673740387, 'learning_rate': 0.00034782608695652176, 'epoch': 0.11}
+
  2%|▏         | 16/750 [01:18<34:34,  2.83s/it]
  2%|▏         | 17/750 [01:21<34:41,  2.84s/it]
                                                
{'loss': 0.3759, 'grad_norm': 0.14612603187561035, 'learning_rate': 0.0003695652173913043, 'epoch': 0.11}
+
  2%|▏         | 17/750 [01:21<34:41,  2.84s/it]
  2%|▏         | 18/750 [01:23<33:31,  2.75s/it]
                                                
{'loss': 0.3598, 'grad_norm': 0.1467358022928238, 'learning_rate': 0.000391304347826087, 'epoch': 0.12}
+
  2%|▏         | 18/750 [01:23<33:31,  2.75s/it]
  3%|▎         | 19/750 [01:26<33:14,  2.73s/it]
                                                
{'loss': 0.3569, 'grad_norm': 0.1298241913318634, 'learning_rate': 0.0004130434782608696, 'epoch': 0.13}
+
  3%|▎         | 19/750 [01:26<33:14,  2.73s/it]
  3%|▎         | 20/750 [01:29<33:49,  2.78s/it]
                                                
{'loss': 0.3642, 'grad_norm': 0.1317041963338852, 'learning_rate': 0.0004347826086956522, 'epoch': 0.13}
+
  3%|▎         | 20/750 [01:29<33:49,  2.78s/it]
  3%|▎         | 21/750 [01:31<31:55,  2.63s/it]
                                                
{'loss': 0.3183, 'grad_norm': 0.14138098061084747, 'learning_rate': 0.00045652173913043474, 'epoch': 0.14}
+
  3%|▎         | 21/750 [01:31<31:55,  2.63s/it]
  3%|▎         | 22/750 [01:34<31:37,  2.61s/it]
                                                
{'loss': 0.3246, 'grad_norm': 0.14274662733078003, 'learning_rate': 0.0004782608695652174, 'epoch': 0.15}
+
  3%|▎         | 22/750 [01:34<31:37,  2.61s/it]
  3%|▎         | 23/750 [01:38<36:12,  2.99s/it]
                                                
{'loss': 0.3166, 'grad_norm': 0.14265847206115723, 'learning_rate': 0.0005, 'epoch': 0.15}
+
  3%|▎         | 23/750 [01:38<36:12,  2.99s/it]
  3%|▎         | 24/750 [01:40<32:58,  2.73s/it]
                                                
{'loss': 0.3272, 'grad_norm': 0.1528300791978836, 'learning_rate': 0.0004894725051862804, 'epoch': 0.16}
+
  3%|▎         | 24/750 [01:40<32:58,  2.73s/it]
  3%|▎         | 25/750 [01:42<31:29,  2.61s/it]
                                                
{'loss': 0.3276, 'grad_norm': 0.14042863249778748, 'learning_rate': 0.00047958315233127195, 'epoch': 0.17}
+
  3%|▎         | 25/750 [01:42<31:29,  2.61s/it]
  3%|▎         | 26/750 [01:44<31:04,  2.57s/it]
                                                
{'loss': 0.3535, 'grad_norm': 0.15493251383304596, 'learning_rate': 0.0004702699715629801, 'epoch': 0.17}
+
  3%|▎         | 26/750 [01:44<31:04,  2.57s/it]
  4%|▎         | 27/750 [01:47<29:55,  2.48s/it]
                                                
{'loss': 0.3188, 'grad_norm': 0.1334412544965744, 'learning_rate': 0.00046147910349544864, 'epoch': 0.18}
+
  4%|▎         | 27/750 [01:47<29:55,  2.48s/it]
  4%|▎         | 28/750 [01:49<29:58,  2.49s/it]
                                                
{'loss': 0.3891, 'grad_norm': 0.17350968718528748, 'learning_rate': 0.00045316348358748285, 'epoch': 0.19}
+
  4%|▎         | 28/750 [01:49<29:58,  2.49s/it]
  4%|▍         | 29/750 [01:52<29:47,  2.48s/it]
                                                
{'loss': 0.3539, 'grad_norm': 0.14614884555339813, 'learning_rate': 0.00044528177828086066, 'epoch': 0.19}
+
  4%|▍         | 29/750 [01:52<29:47,  2.48s/it]
  4%|▍         | 30/750 [01:54<27:35,  2.30s/it]
                                                
{'loss': 0.278, 'grad_norm': 0.12280116230249405, 'learning_rate': 0.0004377975178854565, 'epoch': 0.2}
+
  4%|▍         | 30/750 [01:54<27:35,  2.30s/it]
  4%|▍         | 31/750 [01:58<33:27,  2.79s/it]
                                                
{'loss': 0.318, 'grad_norm': 0.13167476654052734, 'learning_rate': 0.0004306783846070544, 'epoch': 0.21}
+
  4%|▍         | 31/750 [01:58<33:27,  2.79s/it]
  4%|▍         | 32/750 [02:01<36:08,  3.02s/it]
                                                
{'loss': 0.3314, 'grad_norm': 0.12514850497245789, 'learning_rate': 0.00042389562394532925, 'epoch': 0.21}
+
  4%|▍         | 32/750 [02:01<36:08,  3.02s/it]
  4%|▍         | 33/750 [02:03<32:30,  2.72s/it]
                                                
{'loss': 0.331, 'grad_norm': 0.12808619439601898, 'learning_rate': 0.00041742355496836097, 'epoch': 0.22}
+
  4%|▍         | 33/750 [02:03<32:30,  2.72s/it]
  5%|▍         | 34/750 [02:06<34:48,  2.92s/it]
                                                
{'loss': 0.3682, 'grad_norm': 0.14946132898330688, 'learning_rate': 0.00041123916041498716, 'epoch': 0.23}
+
  5%|▍         | 34/750 [02:06<34:48,  2.92s/it]
  5%|▍         | 35/750 [02:09<34:59,  2.94s/it]
                                                
{'loss': 0.2741, 'grad_norm': 0.12048982828855515, 'learning_rate': 0.00040532174168888874, 'epoch': 0.23}
+
  5%|▍         | 35/750 [02:09<34:59,  2.94s/it]
  5%|▍         | 36/750 [02:13<35:42,  3.00s/it]
                                                
{'loss': 0.3375, 'grad_norm': 0.15270358324050903, 'learning_rate': 0.00039965262694272664, 'epoch': 0.24}
+
  5%|▍         | 36/750 [02:13<35:42,  3.00s/it]
  5%|▍         | 37/750 [02:15<33:06,  2.79s/it]
                                                
{'loss': 0.3132, 'grad_norm': 0.1485040932893753, 'learning_rate': 0.00039421492285986, 'epoch': 0.25}
+
  5%|▍         | 37/750 [02:15<33:06,  2.79s/it]
  5%|▌         | 38/750 [02:19<36:39,  3.09s/it]
                                                
{'loss': 0.3534, 'grad_norm': 0.22593602538108826, 'learning_rate': 0.00038899330260774953, 'epoch': 0.25}
+
  5%|▌         | 38/750 [02:19<36:39,  3.09s/it]
  5%|▌         | 39/750 [02:21<32:59,  2.78s/it]
                                                
{'loss': 0.2987, 'grad_norm': 0.1330300122499466, 'learning_rate': 0.00038397382389415225, 'epoch': 0.26}
+
  5%|▌         | 39/750 [02:21<32:59,  2.78s/it]
  5%|▌         | 40/750 [02:24<34:25,  2.91s/it]
                                                
{'loss': 0.2796, 'grad_norm': 0.12396858632564545, 'learning_rate': 0.00037914377220257757, 'epoch': 0.27}
+
  5%|▌         | 40/750 [02:24<34:25,  2.91s/it]
  5%|▌         | 41/750 [02:27<34:46,  2.94s/it]
                                                
{'loss': 0.3473, 'grad_norm': 0.13553141057491302, 'learning_rate': 0.0003744915251898558, 'epoch': 0.27}
+
  5%|▌         | 41/750 [02:27<34:46,  2.94s/it]
  6%|▌         | 42/750 [02:30<33:54,  2.87s/it]
                                                
{'loss': 0.291, 'grad_norm': 0.13075299561023712, 'learning_rate': 0.00037000643495047747, 'epoch': 0.28}
+
  6%|▌         | 42/750 [02:30<33:54,  2.87s/it]
  6%|▌         | 43/750 [02:33<35:38,  3.02s/it]
                                                
{'loss': 0.3316, 'grad_norm': 0.13072647154331207, 'learning_rate': 0.00036567872543061367, 'epoch': 0.29}
+
  6%|▌         | 43/750 [02:33<35:38,  3.02s/it]
  6%|▌         | 44/750 [02:36<35:49,  3.04s/it]
                                                
{'loss': 0.2937, 'grad_norm': 0.13594534993171692, 'learning_rate': 0.0003614994027406106, 'epoch': 0.29}
+
  6%|▌         | 44/750 [02:36<35:49,  3.04s/it]
  6%|▌         | 45/750 [02:39<34:27,  2.93s/it]
                                                
{'loss': 0.2634, 'grad_norm': 0.13940410315990448, 'learning_rate': 0.0003574601764921203, 'epoch': 0.3}
+
  6%|▌         | 45/750 [02:39<34:27,  2.93s/it]
  6%|▌         | 46/750 [02:42<33:36,  2.86s/it]
                                                
{'loss': 0.3113, 'grad_norm': 0.1358165740966797, 'learning_rate': 0.00035355339059327376, 'epoch': 0.31}
+
  6%|▌         | 46/750 [02:42<33:36,  2.86s/it]
  6%|▋         | 47/750 [02:44<31:10,  2.66s/it]
                                                
{'loss': 0.3202, 'grad_norm': 0.1350950002670288, 'learning_rate': 0.0003497719621866722, 'epoch': 0.31}
+
  6%|▋         | 47/750 [02:44<31:10,  2.66s/it]
  6%|▋         | 48/750 [02:47<32:58,  2.82s/it]
                                                
{'loss': 0.3096, 'grad_norm': 0.11780944466590881, 'learning_rate': 0.0003461093276215865, 'epoch': 0.32}
+
  6%|▋         | 48/750 [02:47<32:58,  2.82s/it]
  7%|▋         | 49/750 [02:50<32:45,  2.80s/it]
                                                
{'loss': 0.2998, 'grad_norm': 0.12482839822769165, 'learning_rate': 0.0003425593945223371, 'epoch': 0.33}
+
  7%|▋         | 49/750 [02:50<32:45,  2.80s/it]
  7%|▋         | 50/750 [02:53<34:50,  2.99s/it]
                                                
{'loss': 0.2789, 'grad_norm': 0.10724744945764542, 'learning_rate': 0.0003391164991562634, 'epoch': 0.33}
+
  7%|▋         | 50/750 [02:53<34:50,  2.99s/it]
  7%|▋         | 51/750 [02:55<32:01,  2.75s/it]
                                                
{'loss': 0.2819, 'grad_norm': 0.13204313814640045, 'learning_rate': 0.00033577536842242567, 'epoch': 0.34}
+
  7%|▋         | 51/750 [02:55<32:01,  2.75s/it]
  7%|▋         | 52/750 [02:58<32:13,  2.77s/it]
                                                
{'loss': 0.3037, 'grad_norm': 0.11762496829032898, 'learning_rate': 0.0003325310858805882, 'epoch': 0.35}
+
  7%|▋         | 52/750 [02:58<32:13,  2.77s/it]
  7%|▋         | 53/750 [03:01<31:59,  2.75s/it]
                                                
{'loss': 0.3163, 'grad_norm': 0.126564159989357, 'learning_rate': 0.0003293790613225677, 'epoch': 0.35}
+
  7%|▋         | 53/750 [03:01<31:59,  2.75s/it]
  7%|▋         | 54/750 [03:03<30:00,  2.59s/it]
                                                
{'loss': 0.3215, 'grad_norm': 0.13763472437858582, 'learning_rate': 0.0003263150034575203, 'epoch': 0.36}
+
  7%|▋         | 54/750 [03:03<30:00,  2.59s/it]
  7%|▋         | 55/750 [03:08<39:07,  3.38s/it]
                                                
{'loss': 0.2718, 'grad_norm': 0.11148711293935776, 'learning_rate': 0.00032333489534143163, 'epoch': 0.37}
+
  7%|▋         | 55/750 [03:08<39:07,  3.38s/it]
  7%|▋         | 56/750 [03:12<38:55,  3.37s/it]
                                                
{'loss': 0.2665, 'grad_norm': 0.11790774762630463, 'learning_rate': 0.0003204349722308279, 'epoch': 0.37}
+
  7%|▋         | 56/750 [03:12<38:55,  3.37s/it]
  8%|▊         | 57/750 [03:14<34:39,  3.00s/it]
                                                
{'loss': 0.3249, 'grad_norm': 0.13731083273887634, 'learning_rate': 0.0003176117015830118, 'epoch': 0.38}
+
  8%|▊         | 57/750 [03:14<34:39,  3.00s/it]
  8%|▊         | 58/750 [03:17<36:33,  3.17s/it]
                                                
{'loss': 0.2555, 'grad_norm': 0.12124564498662949, 'learning_rate': 0.00031486176496120126, 'epoch': 0.39}
+
  8%|▊         | 58/750 [03:17<36:33,  3.17s/it]
  8%|▊         | 59/750 [03:24<47:17,  4.11s/it]
                                                
{'loss': 0.2884, 'grad_norm': 0.1108846440911293, 'learning_rate': 0.0003121820416337943, 'epoch': 0.39}
+
  8%|▊         | 59/750 [03:24<47:17,  4.11s/it]
  8%|▊         | 60/750 [03:26<41:43,  3.63s/it]
                                                
{'loss': 0.31, 'grad_norm': 0.1302979737520218, 'learning_rate': 0.00030956959368344517, 'epoch': 0.4}
+
  8%|▊         | 60/750 [03:26<41:43,  3.63s/it]
  8%|▊         | 61/750 [03:29<38:23,  3.34s/it]
                                                
{'loss': 0.3297, 'grad_norm': 0.11964163184165955, 'learning_rate': 0.00030702165246439413, 'epoch': 0.41}
+
  8%|▊         | 61/750 [03:29<38:23,  3.34s/it]
  8%|▊         | 62/750 [03:31<35:31,  3.10s/it]
                                                
{'loss': 0.247, 'grad_norm': 0.10966107249259949, 'learning_rate': 0.0003045356062661163, 'epoch': 0.41}
+
  8%|▊         | 62/750 [03:31<35:31,  3.10s/it]W0627 01:08:10.777487 1454442 site-packages/torch/distributed/run.py:766] 
+W0627 01:08:10.777487 1454442 site-packages/torch/distributed/run.py:766] *****************************************
+W0627 01:08:10.777487 1454442 site-packages/torch/distributed/run.py:766] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+W0627 01:08:10.777487 1454442 site-packages/torch/distributed/run.py:766] *****************************************
+[2025-06-27 01:08:16,530] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-27 01:08:16,560] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-27 01:08:16,565] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-27 01:08:16,596] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-27 01:08:17,977] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-27 01:08:18,055] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-27 01:08:18,058] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-27 01:08:18,100] [INFO] [comm.py:675:init_distributed] cdb=None
+[2025-06-27 01:08:18,100] [INFO] [comm.py:706:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
+[2025-06-27 01:08:18,168] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-27 01:08:18,185] [INFO] [comm.py:675:init_distributed] cdb=None
+[2025-06-27 01:08:18,186] [INFO] [comm.py:675:init_distributed] cdb=None
+[2025-06-27 01:08:18,295] [INFO] [comm.py:675:init_distributed] cdb=None
+06/27/2025 01:08:18 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1distributed training: True, 16-bits training: False
+06/27/2025 01:08:18 - INFO - __main__ - Training/evaluation parameters LoRATrainingArguments(
+_n_gpu=1,
+accelerator_config={'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None, 'use_configured_state': False},
+adafactor=False,
+adam_beta1=0.9,
+adam_beta2=0.999,
+adam_epsilon=1e-08,
+auto_find_batch_size=False,
+batch_eval_metrics=False,
+bf16=True,
+bf16_full_eval=True,
+data_seed=None,
+dataloader_drop_last=False,
+dataloader_num_workers=2,
+dataloader_persistent_workers=False,
+dataloader_pin_memory=True,
+dataloader_prefetch_factor=None,
+ddp_backend=None,
+ddp_broadcast_buffers=None,
+ddp_bucket_cap_mb=None,
+ddp_find_unused_parameters=None,
+ddp_timeout=3600,
+debug=[],
+deepspeed=./config/deepspeed_config.json,
+disable_tqdm=False,
+dispatch_batches=None,
+do_eval=True,
+do_predict=False,
+do_train=True,
+eval_accumulation_steps=None,
+eval_delay=0,
+eval_do_concat_batches=True,
+eval_on_start=True,
+eval_steps=200,
+eval_strategy=steps,
+eval_use_gather_object=False,
+evaluation_strategy=None,
+fp16=False,
+fp16_backend=auto,
+fp16_full_eval=False,
+fp16_opt_level=O1,
+fsdp=[],
+fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False},
+fsdp_min_num_params=0,
+fsdp_transformer_layer_cls_to_wrap=None,
+full_determinism=False,
+gradient_accumulation_steps=1,
+gradient_checkpointing=True,
+gradient_checkpointing_kwargs=None,
+greater_is_better=False,
+group_by_length=False,
+half_precision_backend=auto,
+hub_always_push=False,
+hub_model_id=None,
+hub_private_repo=False,
+hub_strategy=every_save,
+hub_token=<HUB_TOKEN>,
+ignore_data_skip=False,
+include_inputs_for_metrics=False,
+include_num_input_tokens_seen=False,
+include_tokens_per_second=False,
+jit_mode_eval=False,
+label_names=None,
+label_smoothing_factor=0.0,
+learning_rate=0.0005,
+length_column_name=length,
+load_best_model_at_end=True,
+load_lora_from=None,
+local_rank=0,
+log_level=passive,
+log_level_replica=warning,
+log_on_each_node=True,
+logging_dir=./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/runs/Jun27_01-08-16_innmi1srh2-p040,
+logging_first_step=False,
+logging_nan_inf_filter=True,
+logging_steps=1.0,
+logging_strategy=steps,
+lora_config=./config/lora_config.json,
+lr_scheduler_kwargs={},
+lr_scheduler_type=inverse_sqrt,
+max_grad_norm=1.0,
+max_steps=-1,
+metric_for_best_model=eval_loss,
+mp_parameters=,
+neftune_noise_alpha=None,
+no_cuda=False,
+num_train_epochs=5.0,
+optim=adamw_torch,
+optim_args=None,
+optim_target_modules=None,
+output_dir=./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/,
+overwrite_output_dir=True,
+past_index=-1,
+per_device_eval_batch_size=25,
+per_device_train_batch_size=25,
+prediction_loss_only=False,
+push_to_hub=False,
+push_to_hub_model_id=None,
+push_to_hub_organization=None,
+push_to_hub_token=<PUSH_TO_HUB_TOKEN>,
+ray_scope=last,
+remove_unused_columns=True,
+report_to=['wandb'],
+restore_callback_states_from_checkpoint=False,
+resume_from_checkpoint=None,
+run_name=./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/,
+save_on_each_node=False,
+save_only_model=False,
+save_safetensors=True,
+save_steps=200,
+save_strategy=steps,
+save_total_limit=1,
+seed=1,
+skip_memory_metrics=True,
+split_batches=None,
+tf32=None,
+torch_compile=False,
+torch_compile_backend=None,
+torch_compile_mode=None,
+torch_empty_cache_steps=None,
+torchdynamo=None,
+tpu_metrics_debug=False,
+tpu_num_cores=None,
+use_cpu=False,
+use_int8_training=False,
+use_ipex=False,
+use_legacy_prediction_loop=False,
+use_lora=True,
+use_mps_device=False,
+warmup_ratio=0.03,
+warmup_steps=0,
+weight_decay=0.0,
+)
+06/27/2025 01:08:18 - WARNING - __main__ - Process rank: 3, device: cuda:3, n_gpu: 1distributed training: True, 16-bits training: False
+06/27/2025 01:08:18 - WARNING - __main__ - Process rank: 2, device: cuda:2, n_gpu: 1distributed training: True, 16-bits training: False
+06/27/2025 01:08:18 - WARNING - __main__ - Process rank: 1, device: cuda:1, n_gpu: 1distributed training: True, 16-bits training: False
+Using custom data configuration default-0b5518de39c6fcc5
+06/27/2025 01:08:19 - INFO - datasets.builder - Using custom data configuration default-0b5518de39c6fcc5
+Loading Dataset Infos from /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/datasets/packaged_modules/json
+06/27/2025 01:08:19 - INFO - datasets.info - Loading Dataset Infos from /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/datasets/packaged_modules/json
+Overwrite dataset info from restored data version if exists.
+06/27/2025 01:08:19 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists.
+Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+06/27/2025 01:08:19 - INFO - datasets.info - Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+Found cached dataset json (/home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092)
+06/27/2025 01:08:19 - INFO - datasets.builder - Found cached dataset json (/home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092)
+Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+06/27/2025 01:08:19 - INFO - datasets.info - Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+[INFO|configuration_utils.py:733] 2025-06-27 01:08:19,476 >> loading configuration file config.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/config.json
+[INFO|configuration_utils.py:821] 2025-06-27 01:08:19,477 >> Model config LlamaConfig {
+  "_name_or_path": "meta-llama/Llama-3.1-8B-Instruct",
+  "additional_loss_layer": 16,
+  "alignment_matrices_path": null,
+  "apply_inverse": false,
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "contrastive_loss_temperature": 1.0,
+  "contrastive_loss_weight": 1.0,
+  "contrastive_pooling_type": "mean",
+  "distance_function": "cosine",
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "inject_Ws": false,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "only_train_contrastive": false,
+  "only_train_language_modeling": true,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.44.0.dev0",
+  "unidirectional_contrastive_loss": false,
+  "use_cache": true,
+  "vocab_size": 128256
+}
+
+[INFO|tokenization_utils_base.py:2269] 2025-06-27 01:08:19,714 >> loading file tokenizer.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/tokenizer.json
+[INFO|tokenization_utils_base.py:2269] 2025-06-27 01:08:19,714 >> loading file added_tokens.json from cache at None
+[INFO|tokenization_utils_base.py:2269] 2025-06-27 01:08:19,714 >> loading file special_tokens_map.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/special_tokens_map.json
+[INFO|tokenization_utils_base.py:2269] 2025-06-27 01:08:19,714 >> loading file tokenizer_config.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2513] 2025-06-27 01:08:20,027 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
+06/27/2025 01:08:20 - INFO - __main__ - Tokenizer is fast: True
+[INFO|modeling_utils.py:3667] 2025-06-27 01:08:20,031 >> loading weights file model.safetensors from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/model.safetensors.index.json
+[INFO|modeling_utils.py:1591] 2025-06-27 01:08:20,032 >> Instantiating LlamaForCausalLM model under default dtype torch.bfloat16.
+[WARNING|logging.py:328] 2025-06-27 01:08:20,034 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+[INFO|configuration_utils.py:1038] 2025-06-27 01:08:20,036 >> Generate config GenerationConfig {
+  "bos_token_id": 128000,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ]
+}
+
+
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  4.88it/s][WARNING|logging.py:328] 2025-06-27 01:08:20,392 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  6.52it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  7.00it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  4.66it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  7.32it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  6.90it/s]
+[INFO|modeling_utils.py:4499] 2025-06-27 01:08:20,704 >> All model checkpoint weights were used when initializing LlamaForCausalLM.
+
+[INFO|modeling_utils.py:4507] 2025-06-27 01:08:20,705 >> All the weights of LlamaForCausalLM were initialized from the model checkpoint at meta-llama/Llama-3.1-8B-Instruct.
+If your task is similar to the task the model of the checkpoint was trained on, you can already use LlamaForCausalLM for predictions without further training.
+[WARNING|logging.py:328] 2025-06-27 01:08:20,726 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  6.10it/s][INFO|configuration_utils.py:993] 2025-06-27 01:08:20,936 >> loading configuration file generation_config.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/generation_config.json
+[INFO|configuration_utils.py:1038] 2025-06-27 01:08:20,936 >> Generate config GenerationConfig {
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "temperature": 0.6,
+  "top_p": 0.9
+}
+
+adding special tokens...
+06/27/2025 01:08:20 - INFO - __main__ - ================ pad, eos, bos, unk, padding ================ 
+06/27/2025 01:08:20 - INFO - __main__ - <|eot_id|>, 128009
+06/27/2025 01:08:20 - INFO - __main__ - <|eot_id|>, 128009
+06/27/2025 01:08:20 - INFO - __main__ - <|begin_of_text|>, 128000
+06/27/2025 01:08:20 - INFO - __main__ - <|reserved_special_token_0|>, 128002
+06/27/2025 01:08:20 - INFO - __main__ - right
+06/27/2025 01:08:20 - INFO - __main__ - lora_r : 8
+06/27/2025 01:08:20 - INFO - __main__ - lora_alpha : 16
+06/27/2025 01:08:20 - INFO - __main__ - lora_dropout : 0.1
+06/27/2025 01:08:20 - INFO - __main__ - lora_target_modules : ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj']
+06/27/2025 01:08:20 - INFO - __main__ - LoRA configs: LoraConfig(task_type='CAUSAL_LM', peft_type=<PeftType.LORA: 'LORA'>, auto_mapping=None, base_model_name_or_path=None, revision=None, inference_mode=False, r=8, target_modules={'k_proj', 'up_proj', 'v_proj', 'gate_proj', 'down_proj', 'o_proj', 'q_proj'}, exclude_modules=None, lora_alpha=16, lora_dropout=0.1, fan_in_fan_out=False, bias='none', use_rslora=False, modules_to_save=None, init_lora_weights=True, layers_to_transform=None, layers_pattern=None, rank_pattern={}, alpha_pattern={}, megatron_config=None, megatron_core='megatron.core', trainable_token_indices=None, loftq_config={}, eva_config=None, corda_config=None, use_dora=False, layer_replication=None, runtime_config=LoraRuntimeConfig(ephemeral_gpu_offload=False), lora_bias=False)
+
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  3.98it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  4.74it/s][WARNING|logging.py:328] 2025-06-27 01:08:21,115 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  4.95it/s]trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+06/27/2025 01:08:21 - INFO - __main__ - block size: 2048
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  4.86it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  4.95it/s]
+Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-509de0467929df64.arrow
+06/27/2025 01:08:21 - INFO - datasets.arrow_dataset - Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-509de0467929df64.arrow
+
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  5.56it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  3.79it/s]Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-61aa8bb2d831e11e.arrow
+06/27/2025 01:08:21 - INFO - datasets.arrow_dataset - Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-0b5518de39c6fcc5/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-61aa8bb2d831e11e.arrow
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+adding special tokens...
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.99it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.56it/s]
+[rank0]:[W627 01:08:21.375985874 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 0]  using GPU 0 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  4.52it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  5.54it/s]trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank3]:[W627 01:08:21.545564492 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 3]  using GPU 3 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  6.41it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.67it/s]
+adding special tokens...
+adding special tokens...
+trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank2]:[W627 01:08:22.185872698 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 2]  using GPU 2 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank1]:[W627 01:08:22.189653141 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 1]  using GPU 1 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+06/27/2025 01:08:23 - INFO - __main__ - xxx: Showcase the tokenized training samples.
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 2675, 527, 264, 11190, 7033, 26370, 13, 1789, 1475, 3488, 11, 2944, 3094, 555, 3094, 323, 2612, 701, 6425, 304, 264, 2867, 11, 7669, 1697, 3645, 13, 5256, 701, 2077, 449, 330, 8468, 14656, 12, 8468, 22559, 12421, 1501, 682, 29217, 21650, 11, 323, 842, 449, 330, 791, 4320, 374, 1630, 1210, 1405, 1630, 374, 279, 1620, 1121, 13, 2893, 13687, 323, 17879, 304, 701, 33811, 13, 128009, 198, 128006, 882, 128007, 271, 14924, 25, 356, 680, 14912, 15570, 2853, 400, 17, 11, 220, 7725, 30668, 2853, 400, 18, 11, 42030, 326, 1617, 288, 2853, 400, 87, 323, 1560, 1911, 437, 2853, 400, 16, 1855, 13, 39485, 10373, 1063, 21662, 369, 11937, 323, 1063, 4885, 13, 3005, 10373, 2380, 272, 680, 14912, 15570, 11, 1403, 220, 7725, 30668, 11, 1403, 42030, 326, 1617, 288, 11, 323, 1403, 1560, 1911, 437, 13, 2650, 1790, 2349, 1587, 1364, 5371, 1203, 369, 264, 17510, 54135, 4121, 5380, 2746, 584, 1440, 279, 4320, 311, 279, 3485, 3488, 374, 220, 18, 11, 1148, 374, 279, 907, 315, 9987, 3977, 865, 30, 128009, 198, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 8468, 14656, 12, 8468, 22559, 512, 791, 2853, 315, 2380, 272, 680, 14912, 15570, 374, 220, 18, 353, 400, 17, 284, 400, 21, 627, 791, 2853, 315, 1403, 220, 7725, 30668, 374, 220, 17, 353, 400, 18, 284, 400, 21, 627, 791, 2853, 315, 1403, 42030, 326, 1617, 288, 374, 220, 17, 353, 400, 87, 284, 400, 17, 87, 627, 791, 2853, 315, 1403, 1560, 1911, 437, 374, 220, 17, 353, 400, 16, 284, 400, 17, 627, 791, 2860, 2853, 315, 682, 279, 21662, 374, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 627, 50, 13634, 21935, 449, 264, 17510, 54135, 4121, 11, 779, 279, 2349, 1364, 21879, 1203, 374, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 627, 1687, 527, 2728, 430, 279, 2349, 1364, 21879, 1203, 374, 220, 18, 11441, 11, 779, 584, 649, 3350, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 627, 50, 6517, 7922, 11, 584, 636, 25, 400, 17, 87, 284, 400, 18, 627, 12792, 6714, 2225, 11314, 555, 400, 17, 11, 584, 636, 25, 865, 284, 400, 16, 13, 1135, 627, 791, 907, 315, 865, 374, 400, 16, 13, 1135, 627, 791, 4320, 374, 220, 16, 13, 20, 13, 128009]}
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+[INFO|trainer.py:658] 2025-06-27 01:08:25,841 >> Using auto half precision backend
+[2025-06-27 01:08:26,133] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed info: version=0.17.1, git-hash=unknown, git-branch=unknown
+[2025-06-27 01:08:26,133] [INFO] [config.py:655:__init__] Config mesh_device None world_size = 4
+[2025-06-27 01:08:29,798] [INFO] [engine.py:1325:_configure_distributed_model] ********** distributed groups summary **********
+	 self.dp_world_size=4
+	 self.mp_world_size=1
+	 self.seq_dp_world_size=4
+	 self.sequence_parallel_size=1
+***********************************************
+[2025-06-27 01:08:30,762] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Detected CUDA files, patching ldflags
+Emitting ninja build file /home/iitm_admin/.cache/torch_extensions/py39_cu126/cpu_adam/build.ninja...
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/utils/cpp_extension.py:2356: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
+If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
+  warnings.warn(
+Building extension module cpu_adam...
+Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+ninja: no work to do.
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 2.7148239612579346 seconds
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Detected CUDA files, patching ldflags
+Emitting ninja build file /home/iitm_admin/.cache/torch_extensions/py39_cu126/cpu_adam/build.ninja...
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/utils/cpp_extension.py:2356: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
+If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
+  warnings.warn(
+Building extension module cpu_adam...
+Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
+ninja: no work to do.
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 2.7473700046539307 seconds
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 2.770188331604004 seconds
+Adam Optimizer #0 is created with AVX512 arithmetic capability.
+Config: alpha=0.000500, betas=(0.900000, 0.999000), weight_decay=0.000000, adam_w=1
+[2025-06-27 01:08:34,916] [INFO] [logging.py:107:log_dist] [Rank 0] Using DeepSpeed Optimizer param name adam as basic optimizer
+[2025-06-27 01:08:34,916] [INFO] [logging.py:107:log_dist] [Rank 0] Removing param_group that has no 'params' in the basic Optimizer
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 2.8667218685150146 seconds
+[2025-06-27 01:08:34,982] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Basic Optimizer = DeepSpeedCPUAdam
+[2025-06-27 01:08:34,982] [INFO] [utils.py:59:is_zero_supported_optimizer] Checking ZeRO support for optimizer=DeepSpeedCPUAdam type=<class 'deepspeed.ops.adam.cpu_adam.DeepSpeedCPUAdam'>
+[2025-06-27 01:08:34,982] [INFO] [logging.py:107:log_dist] [Rank 0] Creating torch.bfloat16 ZeRO stage 1 optimizer
+[2025-06-27 01:08:34,982] [INFO] [stage_1_and_2.py:151:__init__] Reduce bucket size 200000000
+[2025-06-27 01:08:34,982] [INFO] [stage_1_and_2.py:152:__init__] Allgather bucket size 200000000
+[2025-06-27 01:08:34,983] [INFO] [stage_1_and_2.py:153:__init__] CPU Offload: True
+[2025-06-27 01:08:34,983] [INFO] [stage_1_and_2.py:154:__init__] Round robin gradient partitioning: False
+[2025-06-27 01:08:35,420] [INFO] [utils.py:781:see_memory_usage] Before initializing optimizer states
+[2025-06-27 01:08:35,421] [INFO] [utils.py:782:see_memory_usage] MA 15.0 GB         Max_MA 15.0 GB         CA 15.16 GB         Max_CA 15 GB 
+[2025-06-27 01:08:35,421] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 79.65 GB, percent = 4.0%
+[2025-06-27 01:08:35,599] [INFO] [utils.py:781:see_memory_usage] After initializing optimizer states
+[2025-06-27 01:08:35,599] [INFO] [utils.py:782:see_memory_usage] MA 15.0 GB         Max_MA 15.0 GB         CA 15.16 GB         Max_CA 15 GB 
+[2025-06-27 01:08:35,600] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 79.75 GB, percent = 4.0%
+[2025-06-27 01:08:35,600] [INFO] [stage_1_and_2.py:573:__init__] optimizer state initialized
+[2025-06-27 01:08:35,744] [INFO] [utils.py:781:see_memory_usage] After initializing ZeRO optimizer
+[2025-06-27 01:08:35,744] [INFO] [utils.py:782:see_memory_usage] MA 15.0 GB         Max_MA 15.0 GB         CA 15.16 GB         Max_CA 15 GB 
+[2025-06-27 01:08:35,744] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 79.8 GB, percent = 4.0%
+[2025-06-27 01:08:35,747] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Final Optimizer = DeepSpeedZeroOptimizer
+[2025-06-27 01:08:35,747] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed using client callable to create LR scheduler
+[2025-06-27 01:08:35,747] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed LR Scheduler = <torch.optim.lr_scheduler.LambdaLR object at 0x796bbf71d640>
+[2025-06-27 01:08:35,747] [INFO] [logging.py:107:log_dist] [Rank 0] step=0, skipped=0, lr=[0.0], mom=[[0.9, 0.999]]
+[2025-06-27 01:08:35,753] [INFO] [logging.py:107:log_dist] [Rank 0] [TorchCheckpointEngine] Initialized with serialization = True
+[2025-06-27 01:08:35,753] [INFO] [config.py:921:print] DeepSpeedEngine configuration:
+[2025-06-27 01:08:35,753] [INFO] [config.py:925:print]   activation_checkpointing_config  {
+    "partition_activations": false, 
+    "contiguous_memory_optimization": false, 
+    "cpu_checkpointing": false, 
+    "number_checkpoints": null, 
+    "synchronize_checkpoint_boundary": false, 
+    "profile": false
+}
+[2025-06-27 01:08:35,753] [INFO] [config.py:925:print]   aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'intra_op_parallelism': 1, 'single_submit': False, 'overlap_events': True, 'use_gds': False}
+[2025-06-27 01:08:35,753] [INFO] [config.py:925:print]   amp_enabled .................. False
+[2025-06-27 01:08:35,753] [INFO] [config.py:925:print]   amp_params ................... False
+[2025-06-27 01:08:35,753] [INFO] [config.py:925:print]   autotuning_config ............ {
+    "enabled": false, 
+    "start_step": null, 
+    "end_step": null, 
+    "metric_path": null, 
+    "arg_mappings": null, 
+    "metric": "throughput", 
+    "model_info": null, 
+    "results_dir": "autotuning_results", 
+    "exps_dir": "autotuning_exps", 
+    "overwrite": true, 
+    "fast": true, 
+    "start_profile_step": 3, 
+    "end_profile_step": 5, 
+    "tuner_type": "gridsearch", 
+    "tuner_early_stopping": 5, 
+    "tuner_num_trials": 50, 
+    "model_info_path": null, 
+    "mp_size": 1, 
+    "max_train_batch_size": null, 
+    "min_train_batch_size": 1, 
+    "max_train_micro_batch_size_per_gpu": 1.024000e+03, 
+    "min_train_micro_batch_size_per_gpu": 1, 
+    "num_tuning_micro_batch_sizes": 3
+}
+[2025-06-27 01:08:35,753] [INFO] [config.py:925:print]   bfloat16_config .............. enabled=True immediate_grad_update=False check_grad_overflow=False
+[2025-06-27 01:08:35,753] [INFO] [config.py:925:print]   checkpoint_config ............ {'tag_validation': 'WARN', 'checkpoint_serialization': True, 'writer': None}
+[2025-06-27 01:08:35,753] [INFO] [config.py:925:print]   checkpoint_parallel_write_pipeline  False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   checkpoint_tag_validation_enabled  True
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   checkpoint_tag_validation_fail  False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   comms_config ................. <deepspeed.comm.config.DeepSpeedCommsConfig object at 0x796bc4574cd0>
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   communication_data_type ...... None
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   compile_config ............... deepcompile=False free_activation=False offload_activation=False offload_opt_states=False double_buffer=True symmetric_memory=False debug_log=False offload_parameters=False sync_before_reduce=False sync_after_reduce=False sync_before_allgather=False sync_after_allgather=False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}}
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   curriculum_enabled_legacy .... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   curriculum_params_legacy ..... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'pin_memory': False, 'curriculum_learning': {'enabled': False}, 'dynamic_batching': {'enabled': False, 'lr_scaling_method': 'linear', 'min_batch_size': 1, 'max_batch_size': None, 'sequence_picking_order': 'dataloader', 'verbose': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}}
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   data_efficiency_enabled ...... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   dataloader_drop_last ......... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   disable_allgather ............ False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   dump_state ................... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   eigenvalue_enabled ........... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   eigenvalue_gas_boundary_resolution  1
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   eigenvalue_layer_name ........ bert.encoder.layer
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   eigenvalue_layer_num ......... 0
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   eigenvalue_max_iter .......... 100
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   eigenvalue_stability ......... 1e-06
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   eigenvalue_tol ............... 0.01
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   eigenvalue_verbose ........... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   elasticity_enabled ........... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   float16_config ............... enabled=False auto_cast=False loss_scale=0.0 initial_scale_power=16 loss_scale_window=1000 hysteresis=2 consecutive_hysteresis=False min_loss_scale=1 fp16_master_weights_and_grads=False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   flops_profiler_config ........ {
+    "enabled": false, 
+    "recompute_fwd_factor": 0.0, 
+    "profile_step": 1, 
+    "module_depth": -1, 
+    "top_modules": 1, 
+    "detailed": true, 
+    "output_file": null
+}
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   global_rank .................. 0
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   grad_accum_dtype ............. None
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   gradient_accumulation_steps .. 1
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   gradient_clipping ............ 1.0
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   gradient_predivide_factor .... 1.0
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   graph_harvesting ............. False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   load_universal_checkpoint .... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   memory_breakdown ............. False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   mics_hierarchial_params_gather  False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   mics_shard_size .............. -1
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') comet=CometConfig(enabled=False, samples_log_interval=100, project=None, workspace=None, api_key=None, experiment_name=None, experiment_key=None, online=None, mode=None) wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName')
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   nebula_config ................ {
+    "enabled": false, 
+    "persistent_storage_path": null, 
+    "persistent_time_interval": 100, 
+    "num_of_version_in_retention": 2, 
+    "enable_nebula_load": true, 
+    "load_path": null
+}
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   optimizer_legacy_fusion ...... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   optimizer_name ............... adam
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   optimizer_params ............. {'lr': 0.0005, 'betas': [0.9, 0.999], 'eps': 1e-08, 'weight_decay': 0.0}
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0, 'pipe_partitioned': True, 'grad_partitioned': True}
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   pld_enabled .................. False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   pld_params ................... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   prescale_gradients ........... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   scheduler_name ............... None
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   scheduler_params ............. None
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   seq_parallel_communication_data_type  torch.float32
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   sparse_attention ............. None
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   sparse_gradients_enabled ..... False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   steps_per_print .............. inf
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   tensor_parallel_config ....... dtype=torch.float16 autotp_size=0 tp_overlap_comm=False tensor_parallel=TPConfig(tp_size=1, tp_grain_size=1, mpu=None, tp_group=None) injection_policy_tuple=None keep_module_on_host=False replace_with_kernel_inject=False
+[2025-06-27 01:08:35,754] [INFO] [config.py:925:print]   timers_config ................ enabled=True synchronized=True
+[2025-06-27 01:08:35,755] [INFO] [config.py:925:print]   train_batch_size ............. 100
+[2025-06-27 01:08:35,755] [INFO] [config.py:925:print]   train_micro_batch_size_per_gpu  25
+[2025-06-27 01:08:35,755] [INFO] [config.py:925:print]   use_data_before_expert_parallel_  False
+[2025-06-27 01:08:35,755] [INFO] [config.py:925:print]   use_node_local_storage ....... False
+[2025-06-27 01:08:35,755] [INFO] [config.py:925:print]   wall_clock_breakdown ......... False
+[2025-06-27 01:08:35,755] [INFO] [config.py:925:print]   weight_quantization_config ... None
+[2025-06-27 01:08:35,755] [INFO] [config.py:925:print]   world_size ................... 4
+[2025-06-27 01:08:35,755] [INFO] [config.py:925:print]   zero_allow_untested_optimizer  False
+[2025-06-27 01:08:35,755] [INFO] [config.py:925:print]   zero_config .................. stage=1 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=200000000 use_multi_rank_bucket_allreduce=True allgather_partitions=True allgather_bucket_size=200000000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=None offload_optimizer=DeepSpeedZeroOffloadOptimizerConfig(device='cpu', nvme_path=None, buffer_count=4, pin_memory=True, pipeline_read=False, pipeline_write=False, fast_init=False, ratio=1.0) sub_group_size=1000000000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50000000 param_persistence_threshold=100000 model_persistence_threshold=9223372036854775807 max_live_parameters=1000000000 max_reuse_distance=1000000000 gather_16bit_weights_on_model_save=False module_granularity_threshold=0 use_all_reduce_for_fetch_params=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False zero_hpz_partition_size=1 zero_quantized_weights=False zero_quantized_nontrainable_weights=False zero_quantized_gradients=False zeropp_loco_param=None mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=True pipeline_loading_checkpoint=False override_module_apply=True log_trace_cache_warnings=False
+[2025-06-27 01:08:35,755] [INFO] [config.py:925:print]   zero_enabled ................. True
+[2025-06-27 01:08:35,755] [INFO] [config.py:925:print]   zero_force_ds_cpu_optimizer .. True
+[2025-06-27 01:08:35,755] [INFO] [config.py:925:print]   zero_optimization_stage ...... 1
+[2025-06-27 01:08:35,755] [INFO] [config.py:911:print_user_config]   json = {
+    "optimizer": {
+        "type": "Adam", 
+        "params": {
+            "lr": 0.0005, 
+            "betas": [0.9, 0.999], 
+            "eps": 1e-08, 
+            "weight_decay": 0.0
+        }
+    }, 
+    "bf16": {
+        "enabled": true
+    }, 
+    "fp16": {
+        "enabled": false, 
+        "loss_scale": 0, 
+        "loss_scale_window": 1000, 
+        "initial_scale_power": 16, 
+        "hysteresis": 2, 
+        "min_loss_scale": 1
+    }, 
+    "zero_optimization": {
+        "stage": 1, 
+        "offload_optimizer": {
+            "device": "cpu", 
+            "pin_memory": true
+        }, 
+        "allgather_partitions": true, 
+        "allgather_bucket_size": 2.000000e+08, 
+        "overlap_comm": true, 
+        "reduce_scatter": true, 
+        "reduce_bucket_size": 2.000000e+08, 
+        "contiguous_gradients": true
+    }, 
+    "gradient_accumulation_steps": 1, 
+    "gradient_clipping": 1.0, 
+    "steps_per_print": inf, 
+    "train_batch_size": 100, 
+    "train_micro_batch_size_per_gpu": 25, 
+    "wall_clock_breakdown": false
+}
+[INFO|trainer.py:2145] 2025-06-27 01:08:35,756 >> ***** Running training *****
+[INFO|trainer.py:2146] 2025-06-27 01:08:35,756 >>   Num examples = 15,000
+[INFO|trainer.py:2147] 2025-06-27 01:08:35,756 >>   Num Epochs = 5
+[INFO|trainer.py:2148] 2025-06-27 01:08:35,756 >>   Instantaneous batch size per device = 25
+[INFO|trainer.py:2151] 2025-06-27 01:08:35,756 >>   Total train batch size (w. parallel, distributed & accumulation) = 100
+[INFO|trainer.py:2152] 2025-06-27 01:08:35,756 >>   Gradient Accumulation steps = 1
+[INFO|trainer.py:2153] 2025-06-27 01:08:35,756 >>   Total optimization steps = 750
+[INFO|trainer.py:2154] 2025-06-27 01:08:35,760 >>   Number of trainable parameters = 20,971,520
+[INFO|integration_utils.py:807] 2025-06-27 01:08:35,764 >> Automatic Weights & Biases logging enabled, to disable set os.environ["WANDB_DISABLED"] = "true"
+wandb: WARNING The `run_name` is currently set to the same value as `TrainingArguments.output_dir`. If this was not intended, please specify a different run name by setting the `TrainingArguments.run_name` parameter.
+wandb: Currently logged in as: sidharthpulipaka (indic-encoder) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
+wandb: Tracking run with wandb version 0.20.1
+wandb: Run data is saved locally in /home/iitm_admin/llmteam/mid-align/wandb/run-20250627_010836-dtuhsbl4
+wandb: Run `wandb offline` to turn off syncing.
+wandb: Syncing run ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/
+wandb: ⭐️ View project at https://wandb.ai/indic-encoder/midalign
+wandb: 🚀 View run at https://wandb.ai/indic-encoder/midalign/runs/dtuhsbl4
+
  0%|          | 0/750 [00:00<?, ?it/s][INFO|trainer.py:3831] 2025-06-27 01:08:37,826 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-27 01:08:37,827 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-27 01:08:37,827 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:00<00:03,  2.37it/s]
+
 30%|███       | 3/10 [00:01<00:04,  1.73it/s]
+
 40%|████      | 4/10 [00:02<00:03,  1.61it/s]
+
 50%|█████     | 5/10 [00:04<00:05,  1.18s/it]
+
 60%|██████    | 6/10 [00:05<00:04,  1.06s/it]
+
 70%|███████   | 7/10 [00:06<00:02,  1.07it/s]
+
 80%|████████  | 8/10 [00:06<00:01,  1.22it/s]
+
 90%|█████████ | 9/10 [00:07<00:00,  1.19it/s]
+
100%|██████████| 10/10 [00:10<00:00,  1.50s/it]
                                       
+
                                               
{'eval_loss': 0.5909706354141235, 'eval_accuracy': 0.4538808735936466, 'eval_runtime': 23.2428, 'eval_samples_per_second': 43.024, 'eval_steps_per_second': 0.43, 'epoch': 0}
+
  0%|          | 0/750 [00:23<?, ?it/s]
+
100%|██████████| 10/10 [00:14<00:00,  1.50s/it]
+
                                               
  0%|          | 1/750 [00:34<7:15:05, 34.85s/it]
                                                 
{'loss': 0.6291, 'grad_norm': 0.7676826119422913, 'learning_rate': 2.173913043478261e-05, 'epoch': 0.01}
+
  0%|          | 1/750 [00:34<7:15:05, 34.85s/it]
  0%|          | 2/750 [00:37<3:17:09, 15.81s/it]
                                                 
{'loss': 0.5594, 'grad_norm': 0.6496861577033997, 'learning_rate': 4.347826086956522e-05, 'epoch': 0.01}
+
  0%|          | 2/750 [00:37<3:17:09, 15.81s/it]
  0%|          | 3/750 [00:39<1:58:15,  9.50s/it]
                                                 
{'loss': 0.6411, 'grad_norm': 0.8903798460960388, 'learning_rate': 6.521739130434782e-05, 'epoch': 0.02}
+
  0%|          | 3/750 [00:39<1:58:15,  9.50s/it]
  1%|          | 4/750 [00:42<1:26:06,  6.93s/it]
                                                 
{'loss': 0.6154, 'grad_norm': 0.8014053106307983, 'learning_rate': 8.695652173913044e-05, 'epoch': 0.03}
+
  1%|          | 4/750 [00:42<1:26:06,  6.93s/it]
  1%|          | 5/750 [00:45<1:08:16,  5.50s/it]
                                                 
{'loss': 0.5157, 'grad_norm': 0.9382799863815308, 'learning_rate': 0.00010869565217391305, 'epoch': 0.03}
+
  1%|          | 5/750 [00:45<1:08:16,  5.50s/it]
  1%|          | 6/750 [00:50<1:05:36,  5.29s/it]
                                                 
{'loss': 0.4686, 'grad_norm': 0.6738543510437012, 'learning_rate': 0.00013043478260869564, 'epoch': 0.04}
+
  1%|          | 6/750 [00:50<1:05:36,  5.29s/it]
  1%|          | 7/750 [00:53<58:01,  4.69s/it]  
                                               
{'loss': 0.4484, 'grad_norm': 0.34126800298690796, 'learning_rate': 0.00015217391304347827, 'epoch': 0.05}
+
  1%|          | 7/750 [00:53<58:01,  4.69s/it]
  1%|          | 8/750 [00:55<48:22,  3.91s/it]
                                               
{'loss': 0.4425, 'grad_norm': 0.45595061779022217, 'learning_rate': 0.00017391304347826088, 'epoch': 0.05}
+
  1%|          | 8/750 [00:55<48:22,  3.91s/it]
  1%|          | 9/750 [00:58<44:18,  3.59s/it]
                                               
{'loss': 0.3912, 'grad_norm': 0.35376280546188354, 'learning_rate': 0.0001956521739130435, 'epoch': 0.06}
+
  1%|          | 9/750 [00:58<44:18,  3.59s/it]
  1%|▏         | 10/750 [01:01<39:59,  3.24s/it]
                                                
{'loss': 0.4265, 'grad_norm': 0.36847010254859924, 'learning_rate': 0.0002173913043478261, 'epoch': 0.07}
+
  1%|▏         | 10/750 [01:01<39:59,  3.24s/it]
  1%|▏         | 11/750 [01:03<37:35,  3.05s/it]
                                                
{'loss': 0.3895, 'grad_norm': 0.22510574758052826, 'learning_rate': 0.0002391304347826087, 'epoch': 0.07}
+
  1%|▏         | 11/750 [01:03<37:35,  3.05s/it]
  2%|▏         | 12/750 [01:07<38:39,  3.14s/it]
                                                
{'loss': 0.3748, 'grad_norm': 0.14705240726470947, 'learning_rate': 0.0002608695652173913, 'epoch': 0.08}
+
  2%|▏         | 12/750 [01:07<38:39,  3.14s/it]
  2%|▏         | 13/750 [01:09<37:23,  3.04s/it]
                                                
{'loss': 0.4265, 'grad_norm': 0.1809333860874176, 'learning_rate': 0.0002826086956521739, 'epoch': 0.09}
+
  2%|▏         | 13/750 [01:09<37:23,  3.04s/it]
  2%|▏         | 14/750 [01:12<35:52,  2.92s/it]
                                                
{'loss': 0.3102, 'grad_norm': 0.15456683933734894, 'learning_rate': 0.00030434782608695655, 'epoch': 0.09}
+
  2%|▏         | 14/750 [01:12<35:52,  2.92s/it]
  2%|▏         | 15/750 [01:14<33:43,  2.75s/it]
                                                
{'loss': 0.3844, 'grad_norm': 0.1887345314025879, 'learning_rate': 0.00032608695652173916, 'epoch': 0.1}
+
  2%|▏         | 15/750 [01:14<33:43,  2.75s/it]
  2%|▏         | 16/750 [01:17<34:32,  2.82s/it]
                                                
{'loss': 0.3348, 'grad_norm': 0.13826116919517517, 'learning_rate': 0.00034782608695652176, 'epoch': 0.11}
+
  2%|▏         | 16/750 [01:17<34:32,  2.82s/it]
  2%|▏         | 17/750 [01:20<34:40,  2.84s/it]
                                                
{'loss': 0.376, 'grad_norm': 0.1462388038635254, 'learning_rate': 0.0003695652173913043, 'epoch': 0.11}
+
  2%|▏         | 17/750 [01:20<34:40,  2.84s/it]
  2%|▏         | 18/750 [01:23<33:30,  2.75s/it]
                                                
{'loss': 0.3602, 'grad_norm': 0.14307808876037598, 'learning_rate': 0.000391304347826087, 'epoch': 0.12}
+
  2%|▏         | 18/750 [01:23<33:30,  2.75s/it]
  3%|▎         | 19/750 [01:26<33:11,  2.72s/it]
                                                
{'loss': 0.357, 'grad_norm': 0.12956903874874115, 'learning_rate': 0.0004130434782608696, 'epoch': 0.13}
+
  3%|▎         | 19/750 [01:26<33:11,  2.72s/it]
  3%|▎         | 20/750 [01:28<33:46,  2.78s/it]
                                                
{'loss': 0.364, 'grad_norm': 0.13092580437660217, 'learning_rate': 0.0004347826086956522, 'epoch': 0.13}
+
  3%|▎         | 20/750 [01:28<33:46,  2.78s/it]
  3%|▎         | 21/750 [01:31<31:52,  2.62s/it]
                                                
{'loss': 0.3181, 'grad_norm': 0.14186301827430725, 'learning_rate': 0.00045652173913043474, 'epoch': 0.14}
+
  3%|▎         | 21/750 [01:31<31:52,  2.62s/it]
  3%|▎         | 22/750 [01:33<31:36,  2.61s/it]
                                                
{'loss': 0.3253, 'grad_norm': 0.14143280684947968, 'learning_rate': 0.0004782608695652174, 'epoch': 0.15}
+
  3%|▎         | 22/750 [01:33<31:36,  2.61s/it]
  3%|▎         | 23/750 [01:37<36:14,  2.99s/it]
                                                
{'loss': 0.3168, 'grad_norm': 0.14578765630722046, 'learning_rate': 0.0005, 'epoch': 0.15}
+
  3%|▎         | 23/750 [01:37<36:14,  2.99s/it]
  3%|▎         | 24/750 [01:39<32:59,  2.73s/it]
                                                
{'loss': 0.327, 'grad_norm': 0.15219229459762573, 'learning_rate': 0.0004894725051862804, 'epoch': 0.16}
+
  3%|▎         | 24/750 [01:39<32:59,  2.73s/it]
  3%|▎         | 25/750 [01:42<31:30,  2.61s/it]
                                                
{'loss': 0.3273, 'grad_norm': 0.14175386726856232, 'learning_rate': 0.00047958315233127195, 'epoch': 0.17}
+
  3%|▎         | 25/750 [01:42<31:30,  2.61s/it]
  3%|▎         | 26/750 [01:44<31:08,  2.58s/it]
                                                
{'loss': 0.3537, 'grad_norm': 0.1552947610616684, 'learning_rate': 0.0004702699715629801, 'epoch': 0.17}
+
  3%|▎         | 26/750 [01:44<31:08,  2.58s/it]
  4%|▎         | 27/750 [01:46<29:58,  2.49s/it]
                                                
{'loss': 0.3188, 'grad_norm': 0.13239708542823792, 'learning_rate': 0.00046147910349544864, 'epoch': 0.18}
+
  4%|▎         | 27/750 [01:46<29:58,  2.49s/it]
  4%|▎         | 28/750 [01:49<29:59,  2.49s/it]
                                                
{'loss': 0.3894, 'grad_norm': 0.1757037341594696, 'learning_rate': 0.00045316348358748285, 'epoch': 0.19}
+
  4%|▎         | 28/750 [01:49<29:59,  2.49s/it]
  4%|▍         | 29/750 [01:51<29:48,  2.48s/it]
                                                
{'loss': 0.3542, 'grad_norm': 0.1465374231338501, 'learning_rate': 0.00044528177828086066, 'epoch': 0.19}
+
  4%|▍         | 29/750 [01:51<29:48,  2.48s/it]
  4%|▍         | 30/750 [01:53<27:37,  2.30s/it]
                                                
{'loss': 0.2781, 'grad_norm': 0.12247010320425034, 'learning_rate': 0.0004377975178854565, 'epoch': 0.2}
+
  4%|▍         | 30/750 [01:53<27:37,  2.30s/it]
  4%|▍         | 31/750 [01:57<33:25,  2.79s/it]
                                                
{'loss': 0.3183, 'grad_norm': 0.1324688196182251, 'learning_rate': 0.0004306783846070544, 'epoch': 0.21}
+
  4%|▍         | 31/750 [01:57<33:25,  2.79s/it]
  4%|▍         | 32/750 [02:01<36:12,  3.03s/it]
                                                
{'loss': 0.3311, 'grad_norm': 0.12753865122795105, 'learning_rate': 0.00042389562394532925, 'epoch': 0.21}
+
  4%|▍         | 32/750 [02:01<36:12,  3.03s/it]
  4%|▍         | 33/750 [02:03<32:36,  2.73s/it]
                                                
{'loss': 0.3308, 'grad_norm': 0.13282917439937592, 'learning_rate': 0.00041742355496836097, 'epoch': 0.22}
+
  4%|▍         | 33/750 [02:03<32:36,  2.73s/it]
  5%|▍         | 34/750 [02:06<34:56,  2.93s/it]
                                                
{'loss': 0.3682, 'grad_norm': 0.1494922637939453, 'learning_rate': 0.00041123916041498716, 'epoch': 0.23}
+
  5%|▍         | 34/750 [02:06<34:56,  2.93s/it]
  5%|▍         | 35/750 [02:09<35:06,  2.95s/it]
                                                
{'loss': 0.2742, 'grad_norm': 0.11730839312076569, 'learning_rate': 0.00040532174168888874, 'epoch': 0.23}
+
  5%|▍         | 35/750 [02:09<35:06,  2.95s/it]
  5%|▍         | 36/750 [02:12<35:48,  3.01s/it]
                                                
{'loss': 0.3371, 'grad_norm': 0.1280849725008011, 'learning_rate': 0.00039965262694272664, 'epoch': 0.24}
+
  5%|▍         | 36/750 [02:12<35:48,  3.01s/it]
  5%|▍         | 37/750 [02:15<33:10,  2.79s/it]
                                                
{'loss': 0.3132, 'grad_norm': 0.14156927168369293, 'learning_rate': 0.00039421492285986, 'epoch': 0.25}
+
  5%|▍         | 37/750 [02:15<33:10,  2.79s/it]
  5%|▌         | 38/750 [02:18<36:41,  3.09s/it]
                                                
{'loss': 0.3516, 'grad_norm': 0.12199287116527557, 'learning_rate': 0.00038899330260774953, 'epoch': 0.25}
+
  5%|▌         | 38/750 [02:18<36:41,  3.09s/it]
  5%|▌         | 39/750 [02:20<33:03,  2.79s/it]
                                                
{'loss': 0.2992, 'grad_norm': 0.1307193487882614, 'learning_rate': 0.00038397382389415225, 'epoch': 0.26}
+
  5%|▌         | 39/750 [02:20<33:03,  2.79s/it]
  5%|▌         | 40/750 [02:24<34:30,  2.92s/it]
                                                
{'loss': 0.2798, 'grad_norm': 0.12346825748682022, 'learning_rate': 0.00037914377220257757, 'epoch': 0.27}
+
  5%|▌         | 40/750 [02:24<34:30,  2.92s/it]
  5%|▌         | 41/750 [02:27<34:51,  2.95s/it]
                                                
{'loss': 0.3472, 'grad_norm': 0.13450558483600616, 'learning_rate': 0.0003744915251898558, 'epoch': 0.27}
+
  5%|▌         | 41/750 [02:27<34:51,  2.95s/it]
  6%|▌         | 42/750 [02:29<33:57,  2.88s/it]
                                                
{'loss': 0.2905, 'grad_norm': 0.12511050701141357, 'learning_rate': 0.00037000643495047747, 'epoch': 0.28}
+
  6%|▌         | 42/750 [02:29<33:57,  2.88s/it]
  6%|▌         | 43/750 [02:33<35:37,  3.02s/it]
                                                
{'loss': 0.3315, 'grad_norm': 0.1262725442647934, 'learning_rate': 0.00036567872543061367, 'epoch': 0.29}
+
  6%|▌         | 43/750 [02:33<35:37,  3.02s/it]
  6%|▌         | 44/750 [02:36<35:53,  3.05s/it]
                                                
{'loss': 0.2936, 'grad_norm': 0.132734477519989, 'learning_rate': 0.0003614994027406106, 'epoch': 0.29}
+
  6%|▌         | 44/750 [02:36<35:53,  3.05s/it]
  6%|▌         | 45/750 [02:39<34:31,  2.94s/it]
                                                
{'loss': 0.264, 'grad_norm': 0.13819938898086548, 'learning_rate': 0.0003574601764921203, 'epoch': 0.3}
+
  6%|▌         | 45/750 [02:39<34:31,  2.94s/it]
  6%|▌         | 46/750 [02:41<33:42,  2.87s/it]
                                                
{'loss': 0.3109, 'grad_norm': 0.12718363106250763, 'learning_rate': 0.00035355339059327376, 'epoch': 0.31}
+
  6%|▌         | 46/750 [02:41<33:42,  2.87s/it]
  6%|▋         | 47/750 [02:43<31:13,  2.66s/it]
                                                
{'loss': 0.3196, 'grad_norm': 0.12783795595169067, 'learning_rate': 0.0003497719621866722, 'epoch': 0.31}
+
  6%|▋         | 47/750 [02:43<31:13,  2.66s/it]
  6%|▋         | 48/750 [02:47<32:58,  2.82s/it]
                                                
{'loss': 0.3096, 'grad_norm': 0.11590281128883362, 'learning_rate': 0.0003461093276215865, 'epoch': 0.32}
+
  6%|▋         | 48/750 [02:47<32:58,  2.82s/it]
  7%|▋         | 49/750 [02:49<32:46,  2.81s/it]
                                                
{'loss': 0.2997, 'grad_norm': 0.12182146310806274, 'learning_rate': 0.0003425593945223371, 'epoch': 0.33}
+
  7%|▋         | 49/750 [02:49<32:46,  2.81s/it]
  7%|▋         | 50/750 [02:53<34:51,  2.99s/it]
                                                
{'loss': 0.2788, 'grad_norm': 0.10735758394002914, 'learning_rate': 0.0003391164991562634, 'epoch': 0.33}
+
  7%|▋         | 50/750 [02:53<34:51,  2.99s/it]
  7%|▋         | 51/750 [02:55<32:01,  2.75s/it]
                                                
{'loss': 0.2814, 'grad_norm': 0.1316613405942917, 'learning_rate': 0.00033577536842242567, 'epoch': 0.34}
+
  7%|▋         | 51/750 [02:55<32:01,  2.75s/it]
  7%|▋         | 52/750 [02:58<32:12,  2.77s/it]
                                                
{'loss': 0.304, 'grad_norm': 0.1144656166434288, 'learning_rate': 0.0003325310858805882, 'epoch': 0.35}
+
  7%|▋         | 52/750 [02:58<32:12,  2.77s/it]
  7%|▋         | 53/750 [03:01<31:58,  2.75s/it]
                                                
{'loss': 0.3169, 'grad_norm': 0.12651042640209198, 'learning_rate': 0.0003293790613225677, 'epoch': 0.35}
+
  7%|▋         | 53/750 [03:01<31:58,  2.75s/it]
  7%|▋         | 54/750 [03:03<30:02,  2.59s/it]
                                                
{'loss': 0.3216, 'grad_norm': 0.12947042286396027, 'learning_rate': 0.0003263150034575203, 'epoch': 0.36}
+
  7%|▋         | 54/750 [03:03<30:02,  2.59s/it]
  7%|▋         | 55/750 [03:08<39:03,  3.37s/it]
                                                
{'loss': 0.2713, 'grad_norm': 0.10706916451454163, 'learning_rate': 0.00032333489534143163, 'epoch': 0.37}
+
  7%|▋         | 55/750 [03:08<39:03,  3.37s/it]
  7%|▋         | 56/750 [03:11<38:56,  3.37s/it]
                                                
{'loss': 0.2666, 'grad_norm': 0.11376446485519409, 'learning_rate': 0.0003204349722308279, 'epoch': 0.37}
+
  7%|▋         | 56/750 [03:11<38:56,  3.37s/it]
  8%|▊         | 57/750 [03:13<34:42,  3.01s/it]
                                                
{'loss': 0.325, 'grad_norm': 0.13351619243621826, 'learning_rate': 0.0003176117015830118, 'epoch': 0.38}
+
  8%|▊         | 57/750 [03:14<34:42,  3.01s/it]
  8%|▊         | 58/750 [03:17<36:32,  3.17s/it]
                                                
{'loss': 0.255, 'grad_norm': 0.11766888201236725, 'learning_rate': 0.00031486176496120126, 'epoch': 0.39}
+
  8%|▊         | 58/750 [03:17<36:32,  3.17s/it]
  8%|▊         | 59/750 [03:23<47:17,  4.11s/it]
                                                
{'loss': 0.2882, 'grad_norm': 0.11129848659038544, 'learning_rate': 0.0003121820416337943, 'epoch': 0.39}
+
  8%|▊         | 59/750 [03:23<47:17,  4.11s/it]
  8%|▊         | 60/750 [03:26<41:45,  3.63s/it]
                                                
{'loss': 0.3091, 'grad_norm': 0.12876813113689423, 'learning_rate': 0.00030956959368344517, 'epoch': 0.4}
+
  8%|▊         | 60/750 [03:26<41:45,  3.63s/it]
  8%|▊         | 61/750 [03:29<38:24,  3.34s/it]
                                                
{'loss': 0.3289, 'grad_norm': 0.11419209092855453, 'learning_rate': 0.00030702165246439413, 'epoch': 0.41}
+
  8%|▊         | 61/750 [03:29<38:24,  3.34s/it]
  8%|▊         | 62/750 [03:31<35:31,  3.10s/it]
                                                
{'loss': 0.247, 'grad_norm': 0.10955701023340225, 'learning_rate': 0.0003045356062661163, 'epoch': 0.41}
+
  8%|▊         | 62/750 [03:31<35:31,  3.10s/it]
  8%|▊         | 63/750 [03:37<46:16,  4.04s/it]
                                                
{'loss': 0.3481, 'grad_norm': 0.12455859035253525, 'learning_rate': 0.0003021089890583219, 'epoch': 0.42}
+
  8%|▊         | 63/750 [03:37<46:16,  4.04s/it]
  9%|▊         | 64/750 [03:41<44:02,  3.85s/it]
                                                
{'loss': 0.3178, 'grad_norm': 0.10538862645626068, 'learning_rate': 0.00029973947020704494, 'epoch': 0.43}
+
  9%|▊         | 64/750 [03:41<44:02,  3.85s/it]
  9%|▊         | 65/750 [03:44<40:19,  3.53s/it]
                                                
{'loss': 0.2721, 'grad_norm': 0.10830312967300415, 'learning_rate': 0.0002974248450643262, 'epoch': 0.43}
+
  9%|▊         | 65/750 [03:44<40:19,  3.53s/it]
  9%|▉         | 66/750 [03:46<35:52,  3.15s/it]
                                                
{'loss': 0.2973, 'grad_norm': 0.12905435264110565, 'learning_rate': 0.0002951630263451236, 'epoch': 0.44}
+
  9%|▉         | 66/750 [03:46<35:52,  3.15s/it]
  9%|▉         | 67/750 [03:48<33:06,  2.91s/it]
                                                
{'loss': 0.3063, 'grad_norm': 0.11556871235370636, 'learning_rate': 0.0002929520362147839, 'epoch': 0.45}
+
  9%|▉         | 67/750 [03:48<33:06,  2.91s/it]
  9%|▉         | 68/750 [03:51<32:01,  2.82s/it]
                                                
{'loss': 0.2549, 'grad_norm': 0.11062834411859512, 'learning_rate': 0.00029078999901889984, 'epoch': 0.45}
+
  9%|▉         | 68/750 [03:51<32:01,  2.82s/it]
  9%|▉         | 69/750 [03:53<31:07,  2.74s/it]
                                                
{'loss': 0.2859, 'grad_norm': 0.1299869865179062, 'learning_rate': 0.00028867513459481295, 'epoch': 0.46}
+
  9%|▉         | 69/750 [03:53<31:07,  2.74s/it]
  9%|▉         | 70/750 [03:56<30:23,  2.68s/it]
                                                
{'loss': 0.2487, 'grad_norm': 0.10774589329957962, 'learning_rate': 0.0002866057521105554, 'epoch': 0.47}
+
  9%|▉         | 70/750 [03:56<30:23,  2.68s/it]
  9%|▉         | 71/750 [04:02<41:18,  3.65s/it]
                                                
{'loss': 0.3299, 'grad_norm': 0.13278964161872864, 'learning_rate': 0.0002845802443827711, 'epoch': 0.47}
+
  9%|▉         | 71/750 [04:02<41:18,  3.65s/it]
 10%|▉         | 72/750 [04:05<39:43,  3.52s/it]
                                                
{'loss': 0.2737, 'grad_norm': 0.12108106166124344, 'learning_rate': 0.0002825970826302195, 'epoch': 0.48}
+
 10%|▉         | 72/750 [04:05<39:43,  3.52s/it]
 10%|▉         | 73/750 [04:07<34:26,  3.05s/it]
                                                
{'loss': 0.2826, 'grad_norm': 0.12406221777200699, 'learning_rate': 0.0002806548116239436, 'epoch': 0.49}
+
 10%|▉         | 73/750 [04:07<34:26,  3.05s/it]
 10%|▉         | 74/750 [04:10<34:33,  3.07s/it]
                                                
{'loss': 0.3141, 'grad_norm': 0.1299036741256714, 'learning_rate': 0.00027875204519913876, 'epoch': 0.49}
+
 10%|▉         | 74/750 [04:10<34:33,  3.07s/it]
 10%|█         | 75/750 [04:13<34:02,  3.03s/it]
                                                
{'loss': 0.2888, 'grad_norm': 0.12550075352191925, 'learning_rate': 0.00027688746209726916, 'epoch': 0.5}
+
 10%|█         | 75/750 [04:13<34:02,  3.03s/it]
 10%|█         | 76/750 [04:16<34:21,  3.06s/it]
                                                
{'loss': 0.3173, 'grad_norm': 0.1324121057987213, 'learning_rate': 0.0002750598021100904, 'epoch': 0.51}
+
 10%|█         | 76/750 [04:16<34:21,  3.06s/it]
 10%|█         | 77/750 [04:18<31:35,  2.82s/it]
                                                
{'loss': 0.3159, 'grad_norm': 0.11977514624595642, 'learning_rate': 0.0002732678625000106, 'epoch': 0.51}
+
 10%|█         | 77/750 [04:18<31:35,  2.82s/it]
 10%|█         | 78/750 [04:22<35:42,  3.19s/it]
                                                
{'loss': 0.3225, 'grad_norm': 0.1305081695318222, 'learning_rate': 0.00027151049467368427, 'epoch': 0.52}
+
 10%|█         | 78/750 [04:22<35:42,  3.19s/it]
 11%|█         | 79/750 [04:24<32:02,  2.86s/it]
                                                
{'loss': 0.316, 'grad_norm': 0.1321672946214676, 'learning_rate': 0.00026978660108793815, 'epoch': 0.53}
+
 11%|█         | 79/750 [04:24<32:02,  2.86s/it]
 11%|█         | 80/750 [04:27<32:29,  2.91s/it]
                                                
{'loss': 0.2608, 'grad_norm': 0.1173311397433281, 'learning_rate': 0.00026809513236909023, 'epoch': 0.53}
+
 11%|█         | 80/750 [04:28<32:29,  2.91s/it]
 11%|█         | 81/750 [04:31<32:54,  2.95s/it]
                                                
{'loss': 0.2788, 'grad_norm': 0.09938991814851761, 'learning_rate': 0.00026643508462848443, 'epoch': 0.54}
+
 11%|█         | 81/750 [04:31<32:54,  2.95s/it]
 11%|█         | 82/750 [04:33<32:22,  2.91s/it]
                                                
{'loss': 0.2679, 'grad_norm': 0.13461746275424957, 'learning_rate': 0.00026480549695863983, 'epoch': 0.55}
+
 11%|█         | 82/750 [04:33<32:22,  2.91s/it]
 11%|█         | 83/750 [04:37<33:11,  2.99s/it]
                                                
{'loss': 0.35, 'grad_norm': 0.11837611347436905, 'learning_rate': 0.000263205449095825, 'epoch': 0.55}
+
 11%|█         | 83/750 [04:37<33:11,  2.99s/it]
 11%|█         | 84/750 [04:41<37:19,  3.36s/it]
                                                
{'loss': 0.271, 'grad_norm': 0.11014720797538757, 'learning_rate': 0.0002616340592361418, 'epoch': 0.56}
+
 11%|█         | 84/750 [04:41<37:19,  3.36s/it]
 11%|█▏        | 85/750 [04:44<36:35,  3.30s/it]
                                                
{'loss': 0.289, 'grad_norm': 0.12328944355249405, 'learning_rate': 0.00026009048199334285, 'epoch': 0.57}
+
 11%|█▏        | 85/750 [04:44<36:35,  3.30s/it]
 11%|█▏        | 86/750 [04:47<34:30,  3.12s/it]
                                                
{'loss': 0.3046, 'grad_norm': 0.1270952671766281, 'learning_rate': 0.0002585739064876405, 'epoch': 0.57}
+
 11%|█▏        | 86/750 [04:47<34:30,  3.12s/it]
 12%|█▏        | 87/750 [04:49<30:54,  2.80s/it]
                                                
{'loss': 0.2555, 'grad_norm': 0.1202729269862175, 'learning_rate': 0.0002570835545556902, 'epoch': 0.58}
+
 12%|█▏        | 87/750 [04:49<30:54,  2.80s/it]
 12%|█▏        | 88/750 [04:51<30:16,  2.74s/it]
                                                
{'loss': 0.3009, 'grad_norm': 0.14975877106189728, 'learning_rate': 0.00025561867907277256, 'epoch': 0.59}
+
 12%|█▏        | 88/750 [04:51<30:16,  2.74s/it]
 12%|█▏        | 89/750 [04:54<31:39,  2.87s/it]
                                                
{'loss': 0.2719, 'grad_norm': 0.11323501169681549, 'learning_rate': 0.0002541785623789578, 'epoch': 0.59}
+
 12%|█▏        | 89/750 [04:54<31:39,  2.87s/it]
 12%|█▏        | 90/750 [04:57<30:29,  2.77s/it]
                                                
{'loss': 0.2817, 'grad_norm': 0.1283048391342163, 'learning_rate': 0.00025276251480171836, 'epoch': 0.6}
+
 12%|█▏        | 90/750 [04:57<30:29,  2.77s/it]
 12%|█▏        | 91/750 [05:00<30:47,  2.80s/it]
                                                
{'loss': 0.3195, 'grad_norm': 0.13211776316165924, 'learning_rate': 0.00025136987326808516, 'epoch': 0.61}
+
 12%|█▏        | 91/750 [05:00<30:47,  2.80s/it]
 12%|█▏        | 92/750 [05:03<31:59,  2.92s/it]
                                                
{'loss': 0.3213, 'grad_norm': 0.11941208690404892, 'learning_rate': 0.00025, 'epoch': 0.61}
+
 12%|█▏        | 92/750 [05:03<31:59,  2.92s/it]
 12%|█▏        | 93/750 [05:06<33:07,  3.03s/it]
                                                
{'loss': 0.325, 'grad_norm': 0.11828490346670151, 'learning_rate': 0.00024865228128703607, 'epoch': 0.62}
+
 12%|█▏        | 93/750 [05:06<33:07,  3.03s/it]
 13%|█▎        | 94/750 [05:08<29:42,  2.72s/it]
                                                
{'loss': 0.2685, 'grad_norm': 0.11400855332612991, 'learning_rate': 0.0002473261263311206, 'epoch': 0.63}
+
 13%|█▎        | 94/750 [05:08<29:42,  2.72s/it]
 13%|█▎        | 95/750 [05:11<28:14,  2.59s/it]
                                                
{'loss': 0.2743, 'grad_norm': 0.11376965045928955, 'learning_rate': 0.00024602096615832094, 'epoch': 0.63}
+
 13%|█▎        | 95/750 [05:11<28:14,  2.59s/it]
 13%|█▎        | 96/750 [05:14<29:18,  2.69s/it]
                                                
{'loss': 0.3204, 'grad_norm': 0.11464732140302658, 'learning_rate': 0.0002447362525931402, 'epoch': 0.64}
+
 13%|█▎        | 96/750 [05:14<29:18,  2.69s/it]
 13%|█▎        | 97/750 [05:16<28:07,  2.58s/it]
                                                
{'loss': 0.2738, 'grad_norm': 0.12326445430517197, 'learning_rate': 0.0002434714572911246, 'epoch': 0.65}
+
 13%|█▎        | 97/750 [05:16<28:07,  2.58s/it]
 13%|█▎        | 98/750 [05:19<28:36,  2.63s/it]
                                                
{'loss': 0.2887, 'grad_norm': 0.12062937021255493, 'learning_rate': 0.00024222607082590244, 'epoch': 0.65}
+
 13%|█▎        | 98/750 [05:19<28:36,  2.63s/it]
 13%|█▎        | 99/750 [05:21<28:20,  2.61s/it]
                                                
{'loss': 0.2828, 'grad_norm': 0.12564308941364288, 'learning_rate': 0.00024099960182707373, 'epoch': 0.66}
+
 13%|█▎        | 99/750 [05:21<28:20,  2.61s/it]
 13%|█▎        | 100/750 [05:23<26:22,  2.43s/it]
                                                 
{'loss': 0.2813, 'grad_norm': 0.11408521234989166, 'learning_rate': 0.00023979157616563598, 'epoch': 0.67}
+
 13%|█▎        | 100/750 [05:23<26:22,  2.43s/it]
 13%|█▎        | 101/750 [05:26<28:04,  2.60s/it]
                                                 
{'loss': 0.2957, 'grad_norm': 0.12852343916893005, 'learning_rate': 0.00023860153618387902, 'epoch': 0.67}
+
 13%|█▎        | 101/750 [05:26<28:04,  2.60s/it]
 14%|█▎        | 102/750 [05:28<26:57,  2.50s/it]
                                                 
{'loss': 0.2814, 'grad_norm': 0.11392796039581299, 'learning_rate': 0.00023742903996690855, 'epoch': 0.68}
+
 14%|█▎        | 102/750 [05:28<26:57,  2.50s/it]
 14%|█▎        | 103/750 [05:31<26:20,  2.44s/it]
                                                 
{'loss': 0.2918, 'grad_norm': 0.1252010613679886, 'learning_rate': 0.00023627366065316423, 'epoch': 0.69}
+
 14%|█▎        | 103/750 [05:31<26:20,  2.44s/it]
 14%|█▍        | 104/750 [05:33<27:04,  2.51s/it]
                                                 
{'loss': 0.2986, 'grad_norm': 0.11629941314458847, 'learning_rate': 0.00023513498578149006, 'epoch': 0.69}
+
 14%|█▍        | 104/750 [05:33<27:04,  2.51s/it]
 14%|█▍        | 105/750 [05:36<28:06,  2.61s/it]
                                                 
{'loss': 0.3059, 'grad_norm': 0.115824855864048, 'learning_rate': 0.0002340126166724879, 'epoch': 0.7}
+
 14%|█▍        | 105/750 [05:36<28:06,  2.61s/it]
 14%|█▍        | 106/750 [05:39<27:46,  2.59s/it]
                                                 
{'loss': 0.2954, 'grad_norm': 0.11845732480287552, 'learning_rate': 0.00023290616784204732, 'epoch': 0.71}
+
 14%|█▍        | 106/750 [05:39<27:46,  2.59s/it]
 14%|█▍        | 107/750 [05:41<27:14,  2.54s/it]
                                                 
{'loss': 0.2922, 'grad_norm': 0.11386927217245102, 'learning_rate': 0.00023181526644509276, 'epoch': 0.71}
+
 14%|█▍        | 107/750 [05:41<27:14,  2.54s/it]
 14%|█▍        | 108/750 [05:44<28:06,  2.63s/it]
                                                 
{'loss': 0.349, 'grad_norm': 0.1276896744966507, 'learning_rate': 0.00023073955174772432, 'epoch': 0.72}
+
 14%|█▍        | 108/750 [05:44<28:06,  2.63s/it]
 15%|█▍        | 109/750 [05:47<28:42,  2.69s/it]
                                                 
{'loss': 0.2565, 'grad_norm': 0.11503584682941437, 'learning_rate': 0.0002296786746260559, 'epoch': 0.73}
+
 15%|█▍        | 109/750 [05:47<28:42,  2.69s/it]
 15%|█▍        | 110/750 [05:49<27:06,  2.54s/it]
                                                 
{'loss': 0.3173, 'grad_norm': 0.13803620636463165, 'learning_rate': 0.00022863229709016894, 'epoch': 0.73}
+
 15%|█▍        | 110/750 [05:49<27:06,  2.54s/it]
 15%|█▍        | 111/750 [05:52<27:05,  2.54s/it]
                                                 
{'loss': 0.2782, 'grad_norm': 0.11587632447481155, 'learning_rate': 0.00022760009183170774, 'epoch': 0.74}
+
 15%|█▍        | 111/750 [05:52<27:05,  2.54s/it]
 15%|█▍        | 112/750 [05:54<26:55,  2.53s/it]
                                                 
{'loss': 0.3454, 'grad_norm': 0.12414735555648804, 'learning_rate': 0.00022658174179374143, 'epoch': 0.75}
+
 15%|█▍        | 112/750 [05:54<26:55,  2.53s/it]
 15%|█▌        | 113/750 [05:56<25:02,  2.36s/it]
                                                 
{'loss': 0.2647, 'grad_norm': 0.12405778467655182, 'learning_rate': 0.0002255769397616086, 'epoch': 0.75}
+
 15%|█▌        | 113/750 [05:56<25:02,  2.36s/it]
 15%|█▌        | 114/750 [05:59<25:58,  2.45s/it]
                                                 
{'loss': 0.2585, 'grad_norm': 0.1138877347111702, 'learning_rate': 0.00022458538797354575, 'epoch': 0.76}
+
 15%|█▌        | 114/750 [05:59<25:58,  2.45s/it]
 15%|█▌        | 115/750 [06:01<24:54,  2.35s/it]
                                                 
{'loss': 0.2885, 'grad_norm': 0.12302383780479431, 'learning_rate': 0.00022360679774997898, 'epoch': 0.77}
+
 15%|█▌        | 115/750 [06:01<24:54,  2.35s/it]
 15%|█▌        | 116/750 [06:03<23:25,  2.22s/it]
                                                 
{'loss': 0.2981, 'grad_norm': 0.1360131949186325, 'learning_rate': 0.00022264088914043033, 'epoch': 0.77}
+
 15%|█▌        | 116/750 [06:03<23:25,  2.22s/it]
 16%|█▌        | 117/750 [06:06<27:04,  2.57s/it]
                                                 
{'loss': 0.3159, 'grad_norm': 0.11902552098035812, 'learning_rate': 0.00022168739058705874, 'epoch': 0.78}
+
 16%|█▌        | 117/750 [06:06<27:04,  2.57s/it]
 16%|█▌        | 118/750 [06:09<27:30,  2.61s/it]
                                                 
{'loss': 0.2867, 'grad_norm': 0.11967094242572784, 'learning_rate': 0.00022074603860391704, 'epoch': 0.79}
+
 16%|█▌        | 118/750 [06:09<27:30,  2.61s/it]
 16%|█▌        | 119/750 [06:12<30:34,  2.91s/it]
                                                 
{'loss': 0.2993, 'grad_norm': 0.11197491735219955, 'learning_rate': 0.00021981657747106436, 'epoch': 0.79}
+
 16%|█▌        | 119/750 [06:13<30:34,  2.91s/it]
 16%|█▌        | 120/750 [06:15<28:28,  2.71s/it]
                                                 
{'loss': 0.2893, 'grad_norm': 0.11983267962932587, 'learning_rate': 0.00021889875894272825, 'epoch': 0.8}
+
 16%|█▌        | 120/750 [06:15<28:28,  2.71s/it]
 16%|█▌        | 121/750 [06:17<28:03,  2.68s/it]
                                                 
{'loss': 0.346, 'grad_norm': 0.1266627460718155, 'learning_rate': 0.00021799234196875997, 'epoch': 0.81}
+
 16%|█▌        | 121/750 [06:17<28:03,  2.68s/it]
 16%|█▋        | 122/750 [06:21<31:27,  3.00s/it]
                                                 
{'loss': 0.2811, 'grad_norm': 0.12206675112247467, 'learning_rate': 0.0002170970924286726, 'epoch': 0.81}
+
 16%|█▋        | 122/750 [06:21<31:27,  3.00s/it]
 16%|█▋        | 123/750 [06:23<29:25,  2.82s/it]
                                                 
{'loss': 0.2558, 'grad_norm': 0.11442962288856506, 'learning_rate': 0.00021621278287759676, 'epoch': 0.82}
+
 16%|█▋        | 123/750 [06:23<29:25,  2.82s/it]
 17%|█▋        | 124/750 [06:26<29:28,  2.83s/it]
                                                 
{'loss': 0.2648, 'grad_norm': 0.11016415059566498, 'learning_rate': 0.0002153391923035272, 'epoch': 0.83}
+
 17%|█▋        | 124/750 [06:26<29:28,  2.83s/it]
 17%|█▋        | 125/750 [06:29<29:01,  2.79s/it]
                                                 
{'loss': 0.2917, 'grad_norm': 0.11603953689336777, 'learning_rate': 0.00021447610589527216, 'epoch': 0.83}
+
 17%|█▋        | 125/750 [06:29<29:01,  2.79s/it]
 17%|█▋        | 126/750 [06:35<38:58,  3.75s/it]
                                                 
{'loss': 0.2679, 'grad_norm': 0.1116865873336792, 'learning_rate': 0.0002136233148205519, 'epoch': 0.84}
+
 17%|█▋        | 126/750 [06:35<38:58,  3.75s/it]
 17%|█▋        | 127/750 [06:37<34:22,  3.31s/it]
                                                 
{'loss': 0.3374, 'grad_norm': 0.12836268544197083, 'learning_rate': 0.00021278061601372693, 'epoch': 0.85}
+
 17%|█▋        | 127/750 [06:37<34:22,  3.31s/it]
 17%|█▋        | 128/750 [06:39<29:45,  2.87s/it]
                                                 
{'loss': 0.3614, 'grad_norm': 0.13323533535003662, 'learning_rate': 0.00021194781197266462, 'epoch': 0.85}
+
 17%|█▋        | 128/750 [06:39<29:45,  2.87s/it]
 17%|█▋        | 129/750 [06:42<29:42,  2.87s/it]
                                                 
{'loss': 0.2882, 'grad_norm': 0.10900665074586868, 'learning_rate': 0.00021112471056428407, 'epoch': 0.86}
+
 17%|█▋        | 129/750 [06:42<29:42,  2.87s/it]
 17%|█▋        | 130/750 [06:45<30:47,  2.98s/it]
                                                 
{'loss': 0.3, 'grad_norm': 0.11262764781713486, 'learning_rate': 0.00021031112483834333, 'epoch': 0.87}
+
 17%|█▋        | 130/750 [06:45<30:47,  2.98s/it]
 17%|█▋        | 131/750 [06:48<31:01,  3.01s/it]
                                                 
{'loss': 0.2412, 'grad_norm': 0.10393965989351273, 'learning_rate': 0.00020950687284906043, 'epoch': 0.87}
+
 17%|█▋        | 131/750 [06:48<31:01,  3.01s/it]
 18%|█▊        | 132/750 [06:53<34:54,  3.39s/it]
                                                 
{'loss': 0.315, 'grad_norm': 0.12723898887634277, 'learning_rate': 0.00020871177748418049, 'epoch': 0.88}
+
 18%|█▊        | 132/750 [06:53<34:54,  3.39s/it]
 18%|█▊        | 133/750 [06:56<34:08,  3.32s/it]
                                                 
{'loss': 0.2657, 'grad_norm': 0.13109509646892548, 'learning_rate': 0.00020792566630112533, 'epoch': 0.89}
+
 18%|█▊        | 133/750 [06:56<34:08,  3.32s/it]
 18%|█▊        | 134/750 [06:58<30:37,  2.98s/it]
                                                 
{'loss': 0.2856, 'grad_norm': 0.11769387871026993, 'learning_rate': 0.00020714837136988075, 'epoch': 0.89}
+
 18%|█▊        | 134/750 [06:58<30:37,  2.98s/it]
 18%|█▊        | 135/750 [07:01<29:39,  2.89s/it]
                                                 
{'loss': 0.343, 'grad_norm': 0.1269870549440384, 'learning_rate': 0.00020637972912229674, 'epoch': 0.9}
+
 18%|█▊        | 135/750 [07:01<29:39,  2.89s/it]
 18%|█▊        | 136/750 [07:04<30:04,  2.94s/it]
                                                 
{'loss': 0.2833, 'grad_norm': 0.12557438015937805, 'learning_rate': 0.00020561958020749358, 'epoch': 0.91}
+
 18%|█▊        | 136/750 [07:04<30:04,  2.94s/it]
 18%|█▊        | 137/750 [07:06<28:36,  2.80s/it]
                                                 
{'loss': 0.2937, 'grad_norm': 0.1222861260175705, 'learning_rate': 0.00020486776935308305, 'epoch': 0.91}
+
 18%|█▊        | 137/750 [07:06<28:36,  2.80s/it]
 18%|█▊        | 138/750 [07:09<28:47,  2.82s/it]
                                                 
{'loss': 0.2899, 'grad_norm': 0.11535487323999405, 'learning_rate': 0.00020412414523193154, 'epoch': 0.92}
+
 18%|█▊        | 138/750 [07:09<28:47,  2.82s/it]
 19%|█▊        | 139/750 [07:11<27:14,  2.68s/it]
                                                 
{'loss': 0.3262, 'grad_norm': 0.12347229570150375, 'learning_rate': 0.00020338856033420402, 'epoch': 0.93}
+
 19%|█▊        | 139/750 [07:11<27:14,  2.68s/it]
 19%|█▊        | 140/750 [07:13<24:15,  2.39s/it]
                                                 
{'loss': 0.3038, 'grad_norm': 0.13343371450901031, 'learning_rate': 0.00020266087084444437, 'epoch': 0.93}
+
 19%|█▊        | 140/750 [07:13<24:15,  2.39s/it]
 19%|█▉        | 141/750 [07:16<24:15,  2.39s/it]
                                                 
{'loss': 0.2854, 'grad_norm': 0.12090680003166199, 'learning_rate': 0.00020194093652345886, 'epoch': 0.94}
+
 19%|█▉        | 141/750 [07:16<24:15,  2.39s/it]
 19%|█▉        | 142/750 [07:18<24:18,  2.40s/it]
                                                 
{'loss': 0.2677, 'grad_norm': 0.12249066680669785, 'learning_rate': 0.00020122862059478237, 'epoch': 0.95}
+
 19%|█▉        | 142/750 [07:18<24:18,  2.40s/it]
 19%|█▉        | 143/750 [07:20<24:20,  2.41s/it]
                                                 
{'loss': 0.3035, 'grad_norm': 0.11707831174135208, 'learning_rate': 0.00020052378963551984, 'epoch': 0.95}
+
 19%|█▉        | 143/750 [07:20<24:20,  2.41s/it]
 19%|█▉        | 144/750 [07:23<23:59,  2.37s/it]
                                                 
{'loss': 0.2796, 'grad_norm': 0.11223620921373367, 'learning_rate': 0.00019982631347136332, 'epoch': 0.96}
+
 19%|█▉        | 144/750 [07:23<23:59,  2.37s/it]
 19%|█▉        | 145/750 [07:25<24:04,  2.39s/it]
                                                 
{'loss': 0.2877, 'grad_norm': 0.11740516126155853, 'learning_rate': 0.00019913606507559878, 'epoch': 0.97}
+
 19%|█▉        | 145/750 [07:25<24:04,  2.39s/it]
 19%|█▉        | 146/750 [07:28<25:54,  2.57s/it]
                                                 
{'loss': 0.2555, 'grad_norm': 0.10738462954759598, 'learning_rate': 0.00019845292047192357, 'epoch': 0.97}
+
 19%|█▉        | 146/750 [07:28<25:54,  2.57s/it]
 20%|█▉        | 147/750 [07:31<25:48,  2.57s/it]
                                                 
{'loss': 0.289, 'grad_norm': 0.10919294506311417, 'learning_rate': 0.00019777675864090656, 'epoch': 0.98}
+
 20%|█▉        | 147/750 [07:31<25:48,  2.57s/it]
 20%|█▉        | 148/750 [07:33<26:09,  2.61s/it]
                                                 
{'loss': 0.3172, 'grad_norm': 0.11614782363176346, 'learning_rate': 0.00019710746142993, 'epoch': 0.99}
+
 20%|█▉        | 148/750 [07:33<26:09,  2.61s/it]
 20%|█▉        | 149/750 [07:36<25:26,  2.54s/it]
                                                 
{'loss': 0.2867, 'grad_norm': 0.1248602643609047, 'learning_rate': 0.00019644491346646172, 'epoch': 0.99}
+
 20%|█▉        | 149/750 [07:36<25:26,  2.54s/it]
 20%|██        | 150/750 [07:41<34:45,  3.48s/it]
                                                 
{'loss': 0.3085, 'grad_norm': 0.11233309656381607, 'learning_rate': 0.00019578900207451217, 'epoch': 1.0}
+
 20%|██        | 150/750 [07:41<34:45,  3.48s/it]
 20%|██        | 151/750 [07:54<1:01:25,  6.15s/it]
                                                   
{'loss': 0.2203, 'grad_norm': 0.10600850731134415, 'learning_rate': 0.00019513961719414, 'epoch': 1.01}
+
 20%|██        | 151/750 [07:54<1:01:25,  6.15s/it]
 20%|██        | 152/750 [07:56<49:36,  4.98s/it]  
                                                 
{'loss': 0.2209, 'grad_norm': 0.10490038245916367, 'learning_rate': 0.00019449665130387477, 'epoch': 1.01}
+
 20%|██        | 152/750 [07:56<49:36,  4.98s/it]
 20%|██        | 153/750 [07:59<44:14,  4.45s/it]
                                                 
{'loss': 0.2305, 'grad_norm': 0.1184036061167717, 'learning_rate': 0.0001938599993459332, 'epoch': 1.02}
+
 20%|██        | 153/750 [07:59<44:14,  4.45s/it]
 21%|██        | 154/750 [08:02<38:49,  3.91s/it]
                                                 
{'loss': 0.3143, 'grad_norm': 0.13581165671348572, 'learning_rate': 0.00019322955865411052, 'epoch': 1.03}
+
 21%|██        | 154/750 [08:02<38:49,  3.91s/it]
 21%|██        | 155/750 [08:04<33:46,  3.41s/it]
                                                 
{'loss': 0.2462, 'grad_norm': 0.12197927385568619, 'learning_rate': 0.00019260522888423458, 'epoch': 1.03}
+
 21%|██        | 155/750 [08:04<33:46,  3.41s/it]
 21%|██        | 156/750 [08:07<31:58,  3.23s/it]
                                                 
{'loss': 0.239, 'grad_norm': 0.11918488144874573, 'learning_rate': 0.00019198691194707612, 'epoch': 1.04}
+
 21%|██        | 156/750 [08:07<31:58,  3.23s/it]
 21%|██        | 157/750 [08:09<28:59,  2.93s/it]
                                                 
{'loss': 0.2527, 'grad_norm': 0.12025479972362518, 'learning_rate': 0.00019137451194361295, 'epoch': 1.05}
+
 21%|██        | 157/750 [08:09<28:59,  2.93s/it]
 21%|██        | 158/750 [08:12<27:45,  2.81s/it]
                                                 
{'loss': 0.2241, 'grad_norm': 0.11809056997299194, 'learning_rate': 0.00019076793510255105, 'epoch': 1.05}
+
 21%|██        | 158/750 [08:12<27:45,  2.81s/it]
 21%|██        | 159/750 [08:15<30:04,  3.05s/it]
                                                 
{'loss': 0.2912, 'grad_norm': 0.13311469554901123, 'learning_rate': 0.00019016708972001076, 'epoch': 1.06}
+
 21%|██        | 159/750 [08:15<30:04,  3.05s/it]
 21%|██▏       | 160/750 [08:19<31:57,  3.25s/it]
                                                 
{'loss': 0.2916, 'grad_norm': 0.119200199842453, 'learning_rate': 0.00018957188610128878, 'epoch': 1.07}
+
 21%|██▏       | 160/750 [08:19<31:57,  3.25s/it]
 21%|██▏       | 161/750 [08:23<33:37,  3.43s/it]
                                                 
{'loss': 0.2118, 'grad_norm': 0.12363266199827194, 'learning_rate': 0.0001889822365046136, 'epoch': 1.07}
+
 21%|██▏       | 161/750 [08:23<33:37,  3.43s/it]
 22%|██▏       | 162/750 [08:25<30:59,  3.16s/it]
                                                 
{'loss': 0.2792, 'grad_norm': 0.1319878101348877, 'learning_rate': 0.00018839805508681298, 'epoch': 1.08}
+
 22%|██▏       | 162/750 [08:25<30:59,  3.16s/it]
 22%|██▏       | 163/750 [08:28<28:14,  2.89s/it]
                                                 
{'loss': 0.2755, 'grad_norm': 0.13106536865234375, 'learning_rate': 0.00018781925785081761, 'epoch': 1.09}
+
 22%|██▏       | 163/750 [08:28<28:14,  2.89s/it]
 22%|██▏       | 164/750 [08:30<25:37,  2.62s/it]
                                                 
{'loss': 0.2583, 'grad_norm': 0.12672126293182373, 'learning_rate': 0.0001872457625949279, 'epoch': 1.09}
+
 22%|██▏       | 164/750 [08:30<25:37,  2.62s/it]
 22%|██▏       | 165/750 [08:33<26:15,  2.69s/it]
                                                 
{'loss': 0.2584, 'grad_norm': 0.1290132850408554, 'learning_rate': 0.00018667748886377504, 'epoch': 1.1}
+
 22%|██▏       | 165/750 [08:33<26:15,  2.69s/it]
 22%|██▏       | 166/750 [08:35<25:03,  2.57s/it]
                                                 
{'loss': 0.2474, 'grad_norm': 0.11362136155366898, 'learning_rate': 0.00018611435790090854, 'epoch': 1.11}
+
 22%|██▏       | 166/750 [08:35<25:03,  2.57s/it]
 22%|██▏       | 167/750 [08:37<24:37,  2.53s/it]
                                                 
{'loss': 0.264, 'grad_norm': 0.13249048590660095, 'learning_rate': 0.00018555629260294813, 'epoch': 1.11}
+
 22%|██▏       | 167/750 [08:37<24:37,  2.53s/it]
 22%|██▏       | 168/750 [08:40<25:59,  2.68s/it]
                                                 
{'loss': 0.2577, 'grad_norm': 0.13223296403884888, 'learning_rate': 0.00018500321747523874, 'epoch': 1.12}
+
 22%|██▏       | 168/750 [08:40<25:59,  2.68s/it]
 23%|██▎       | 169/750 [08:43<24:59,  2.58s/it]
                                                 
{'loss': 0.2214, 'grad_norm': 0.1194254532456398, 'learning_rate': 0.00018445505858895076, 'epoch': 1.13}
+
 23%|██▎       | 169/750 [08:43<24:59,  2.58s/it]
 23%|██▎       | 170/750 [08:45<24:59,  2.59s/it]
                                                 
{'loss': 0.299, 'grad_norm': 0.13304321467876434, 'learning_rate': 0.0001839117435395704, 'epoch': 1.13}
+
 23%|██▎       | 170/750 [08:45<24:59,  2.59s/it]
 23%|██▎       | 171/750 [08:48<24:47,  2.57s/it]
                                                 
{'loss': 0.2355, 'grad_norm': 0.1287664771080017, 'learning_rate': 0.00018337320140672696, 'epoch': 1.14}
+
 23%|██▎       | 171/750 [08:48<24:47,  2.57s/it]
 23%|██▎       | 172/750 [08:50<25:12,  2.62s/it]
                                                 
{'loss': 0.3202, 'grad_norm': 0.14604395627975464, 'learning_rate': 0.00018283936271530683, 'epoch': 1.15}
+
 23%|██▎       | 172/750 [08:50<25:12,  2.62s/it]
 23%|██▎       | 173/750 [08:54<27:39,  2.88s/it]
                                                 
{'loss': 0.3125, 'grad_norm': 0.13260531425476074, 'learning_rate': 0.0001823101593978053, 'epoch': 1.15}
+
 23%|██▎       | 173/750 [08:54<27:39,  2.88s/it]
 23%|██▎       | 174/750 [08:57<27:33,  2.87s/it]
                                                 
{'loss': 0.2237, 'grad_norm': 0.13792310655117035, 'learning_rate': 0.00018178552475787025, 'epoch': 1.16}
+
 23%|██▎       | 174/750 [08:57<27:33,  2.87s/it]
 23%|██▎       | 175/750 [08:59<26:53,  2.81s/it]
                                                 
{'loss': 0.2735, 'grad_norm': 0.13103075325489044, 'learning_rate': 0.00018126539343499314, 'epoch': 1.17}
+
 23%|██▎       | 175/750 [08:59<26:53,  2.81s/it]
 23%|██▎       | 176/750 [09:01<24:17,  2.54s/it]
                                                 
{'loss': 0.2106, 'grad_norm': 0.1255270540714264, 'learning_rate': 0.0001807497013703053, 'epoch': 1.17}
+
 23%|██▎       | 176/750 [09:01<24:17,  2.54s/it]
 24%|██▎       | 177/750 [09:04<24:54,  2.61s/it]
                                                 
{'loss': 0.2636, 'grad_norm': 0.13411873579025269, 'learning_rate': 0.00018023838577343806, 'epoch': 1.18}
+
 24%|██▎       | 177/750 [09:04<24:54,  2.61s/it]
 24%|██▎       | 178/750 [09:07<26:36,  2.79s/it]
                                                 
{'loss': 0.2595, 'grad_norm': 0.14472921192646027, 'learning_rate': 0.0001797313850904089, 'epoch': 1.19}
+
 24%|██▎       | 178/750 [09:07<26:36,  2.79s/it]
 24%|██▍       | 179/750 [09:11<27:44,  2.92s/it]
                                                 
{'loss': 0.2556, 'grad_norm': 0.12807022035121918, 'learning_rate': 0.00017922863897249503, 'epoch': 1.19}
+
 24%|██▍       | 179/750 [09:11<27:44,  2.92s/it]
 24%|██▍       | 180/750 [09:14<28:07,  2.96s/it]
                                                 
{'loss': 0.2296, 'grad_norm': 0.13180653750896454, 'learning_rate': 0.00017873008824606016, 'epoch': 1.2}
+
 24%|██▍       | 180/750 [09:14<28:07,  2.96s/it]
 24%|██▍       | 181/750 [09:18<32:48,  3.46s/it]
                                                 
{'loss': 0.2676, 'grad_norm': 0.14499099552631378, 'learning_rate': 0.00017823567488329874, 'epoch': 1.21}
+
 24%|██▍       | 181/750 [09:18<32:48,  3.46s/it]
 24%|██▍       | 182/750 [09:20<29:16,  3.09s/it]
                                                 
{'loss': 0.2392, 'grad_norm': 0.14604854583740234, 'learning_rate': 0.00017774534197386607, 'epoch': 1.21}
+
 24%|██▍       | 182/750 [09:21<29:16,  3.09s/it]
 24%|██▍       | 183/750 [09:23<27:03,  2.86s/it]
                                                 
{'loss': 0.224, 'grad_norm': 0.1286548525094986, 'learning_rate': 0.0001772590336973617, 'epoch': 1.22}
+
 24%|██▍       | 183/750 [09:23<27:03,  2.86s/it]
 25%|██▍       | 184/750 [09:26<27:33,  2.92s/it]
                                                 
{'loss': 0.2572, 'grad_norm': 0.13301284611225128, 'learning_rate': 0.00017677669529663688, 'epoch': 1.23}
+
 25%|██▍       | 184/750 [09:26<27:33,  2.92s/it]
 25%|██▍       | 185/750 [09:30<31:48,  3.38s/it]
                                                 
{'loss': 0.2159, 'grad_norm': 0.13925692439079285, 'learning_rate': 0.00017629827305189658, 'epoch': 1.23}
+
 25%|██▍       | 185/750 [09:30<31:48,  3.38s/it]
 25%|██▍       | 186/750 [09:32<27:49,  2.96s/it]
                                                 
{'loss': 0.2525, 'grad_norm': 0.15350250899791718, 'learning_rate': 0.0001758237142555681, 'epoch': 1.24}
+
 25%|██▍       | 186/750 [09:32<27:49,  2.96s/it]
 25%|██▍       | 187/750 [09:36<28:28,  3.04s/it]
                                                 
{'loss': 0.3002, 'grad_norm': 0.1524312049150467, 'learning_rate': 0.00017535296718791012, 'epoch': 1.25}
+
 25%|██▍       | 187/750 [09:36<28:28,  3.04s/it]
 25%|██▌       | 188/750 [09:38<27:50,  2.97s/it]
                                                 
{'loss': 0.2579, 'grad_norm': 0.14768707752227783, 'learning_rate': 0.0001748859810933361, 'epoch': 1.25}
+
 25%|██▌       | 188/750 [09:38<27:50,  2.97s/it]
 25%|██▌       | 189/750 [09:41<26:37,  2.85s/it]
                                                 
{'loss': 0.2343, 'grad_norm': 0.12535709142684937, 'learning_rate': 0.00017442270615742787, 'epoch': 1.26}
+
 25%|██▌       | 189/750 [09:41<26:37,  2.85s/it]
 25%|██▌       | 190/750 [09:43<25:11,  2.70s/it]
                                                 
{'loss': 0.2338, 'grad_norm': 0.14354152977466583, 'learning_rate': 0.00017396309348461487, 'epoch': 1.27}
+
 25%|██▌       | 190/750 [09:43<25:11,  2.70s/it]
 25%|██▌       | 191/750 [09:46<26:33,  2.85s/it]
                                                 
{'loss': 0.2602, 'grad_norm': 0.14885756373405457, 'learning_rate': 0.00017350709507649772, 'epoch': 1.27}
+
 25%|██▌       | 191/750 [09:46<26:33,  2.85s/it]
 26%|██▌       | 192/750 [09:49<25:41,  2.76s/it]
                                                 
{'loss': 0.2534, 'grad_norm': 0.14696653187274933, 'learning_rate': 0.00017305466381079324, 'epoch': 1.28}
+
 26%|██▌       | 192/750 [09:49<25:41,  2.76s/it]
 26%|██▌       | 193/750 [09:51<24:45,  2.67s/it]
                                                 
{'loss': 0.2614, 'grad_norm': 0.1386875957250595, 'learning_rate': 0.00017260575342088004, 'epoch': 1.29}
+
 26%|██▌       | 193/750 [09:51<24:45,  2.67s/it]
 26%|██▌       | 194/750 [09:53<22:25,  2.42s/it]
                                                 
{'loss': 0.2436, 'grad_norm': 0.1448613554239273, 'learning_rate': 0.0001721603184759251, 'epoch': 1.29}
+
 26%|██▌       | 194/750 [09:53<22:25,  2.42s/it]
 26%|██▌       | 195/750 [09:56<24:08,  2.61s/it]
                                                 
{'loss': 0.2652, 'grad_norm': 0.1558540314435959, 'learning_rate': 0.0001717183143615715, 'epoch': 1.3}
+
 26%|██▌       | 195/750 [09:56<24:08,  2.61s/it]
 26%|██▌       | 196/750 [09:59<25:15,  2.73s/it]
                                                 
{'loss': 0.169, 'grad_norm': 0.11521285027265549, 'learning_rate': 0.00017127969726116855, 'epoch': 1.31}
+
 26%|██▌       | 196/750 [09:59<25:15,  2.73s/it]
 26%|██▋       | 197/750 [10:02<25:09,  2.73s/it]
                                                 
{'loss': 0.2327, 'grad_norm': 0.13666842877864838, 'learning_rate': 0.0001708444241375271, 'epoch': 1.31}
+
 26%|██▋       | 197/750 [10:02<25:09,  2.73s/it]
 26%|██▋       | 198/750 [10:05<26:25,  2.87s/it]
                                                 
{'loss': 0.2582, 'grad_norm': 0.14269359409809113, 'learning_rate': 0.00017041245271518172, 'epoch': 1.32}
+
 26%|██▋       | 198/750 [10:05<26:25,  2.87s/it]
 27%|██▋       | 199/750 [10:08<25:44,  2.80s/it]
                                                 
{'loss': 0.236, 'grad_norm': 0.13477157056331635, 'learning_rate': 0.00016998374146314418, 'epoch': 1.33}
+
 27%|██▋       | 199/750 [10:08<25:44,  2.80s/it]
 27%|██▋       | 200/750 [10:10<23:48,  2.60s/it]
                                                 
{'loss': 0.2403, 'grad_norm': 0.1597282886505127, 'learning_rate': 0.0001695582495781317, 'epoch': 1.33}
+
 27%|██▋       | 200/750 [10:10<23:48,  2.60s/it][INFO|trainer.py:3831] 2025-06-27 01:18:48,394 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-27 01:18:48,394 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-27 01:18:48,394 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:00<00:03,  2.44it/s]
+
 30%|███       | 3/10 [00:01<00:03,  1.75it/s]
+
 40%|████      | 4/10 [00:02<00:03,  1.62it/s]
+
 50%|█████     | 5/10 [00:03<00:04,  1.20it/s]
+
 60%|██████    | 6/10 [00:04<00:03,  1.21it/s]
+
 70%|███████   | 7/10 [00:05<00:02,  1.29it/s]
+
 80%|████████  | 8/10 [00:05<00:01,  1.41it/s]
+
 90%|█████████ | 9/10 [00:06<00:00,  1.30it/s]
+
100%|██████████| 10/10 [00:09<00:00,  1.49s/it]
                                                 
+
                                               
{'eval_loss': 0.2748406231403351, 'eval_accuracy': 0.46148841826604897, 'eval_runtime': 22.8697, 'eval_samples_per_second': 43.726, 'eval_steps_per_second': 0.437, 'epoch': 1.33}
+
 27%|██▋       | 200/750 [10:33<23:48,  2.60s/it]
+
100%|██████████| 10/10 [00:13<00:00,  1.49s/it]
+
                                               /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[INFO|trainer.py:3515] 2025-06-27 01:19:25,353 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-200
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da445-05a0be63165b35303c2adeaf;3aa4162a-9483-4406-9c68-8a897189b40d)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-27 01:19:25,701 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-200/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-27 01:19:25,702 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-200/special_tokens_map.json
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[2025-06-27 01:19:26,791] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Checkpoint global_step200 is begin to save!
+[2025-06-27 01:19:26,817] [INFO] [logging.py:107:log_dist] [Rank 0] Saving model checkpoint: ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-200/global_step200/mp_rank_00_model_states.pt
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da447-6fa5746842d1eb50291e7106;746030c8-0199-4b8b-9a43-5d4fed2b3994)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da447-4c73618d7bd94fed6bb33b84;5126b720-1390-46b4-82a3-5ed0f7b04eb9)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da447-737fec841f6beb386616d397;042edbc5-24d6-427b-b431-ea8ba33a32b4)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da447-17b6c217578f57bf0c924690;e9742f95-056f-4525-9187-03d78e0c75ff)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+
 27%|██▋       | 201/750 [10:52<2:11:40, 14.39s/it]
                                                   
{'loss': 0.2624, 'grad_norm': 0.14358873665332794, 'learning_rate': 0.0001691359369682545, 'epoch': 1.34}
+
 27%|██▋       | 201/750 [10:52<2:11:40, 14.39s/it]
 27%|██▋       | 202/750 [10:54<1:37:28, 10.67s/it]
                                                   
{'loss': 0.2177, 'grad_norm': 0.1422012448310852, 'learning_rate': 0.00016871676423714827, 'epoch': 1.35}
+
 27%|██▋       | 202/750 [10:54<1:37:28, 10.67s/it]
 27%|██▋       | 203/750 [10:57<1:16:04,  8.34s/it]
                                                   
{'loss': 0.263, 'grad_norm': 0.1402103155851364, 'learning_rate': 0.00016830069266853705, 'epoch': 1.35}
+
 27%|██▋       | 203/750 [10:57<1:16:04,  8.34s/it]
 27%|██▋       | 204/750 [11:00<1:02:06,  6.83s/it]
                                                   
{'loss': 0.2724, 'grad_norm': 0.1454554945230484, 'learning_rate': 0.00016788768421121283, 'epoch': 1.36}
+
 27%|██▋       | 204/750 [11:00<1:02:06,  6.83s/it]
 27%|██▋       | 205/750 [11:03<51:45,  5.70s/it]  
                                                 
{'loss': 0.2529, 'grad_norm': 0.1449185162782669, 'learning_rate': 0.00016747770146441848, 'epoch': 1.37}
+
 27%|██▋       | 205/750 [11:03<51:45,  5.70s/it]
 27%|██▋       | 206/750 [11:05<42:05,  4.64s/it]
                                                 
{'loss': 0.247, 'grad_norm': 0.14889460802078247, 'learning_rate': 0.0001670707076636216, 'epoch': 1.37}
+
 27%|██▋       | 206/750 [11:05<42:05,  4.64s/it]
 28%|██▊       | 207/750 [11:08<35:26,  3.92s/it]
                                                 
{'loss': 0.2609, 'grad_norm': 0.15383079648017883, 'learning_rate': 0.00016666666666666666, 'epoch': 1.38}
+
 28%|██▊       | 207/750 [11:08<35:26,  3.92s/it]
 28%|██▊       | 208/750 [11:10<31:55,  3.53s/it]
                                                 
{'loss': 0.2427, 'grad_norm': 0.1365276426076889, 'learning_rate': 0.0001662655429402941, 'epoch': 1.39}
+
 28%|██▊       | 208/750 [11:10<31:55,  3.53s/it]
 28%|██▊       | 209/750 [11:14<31:47,  3.53s/it]
                                                 
{'loss': 0.2231, 'grad_norm': 0.13187739253044128, 'learning_rate': 0.00016586730154701388, 'epoch': 1.39}
+
 28%|██▊       | 209/750 [11:14<31:47,  3.53s/it]
 28%|██▊       | 210/750 [11:17<30:18,  3.37s/it]
                                                 
{'loss': 0.2529, 'grad_norm': 0.15917284786701202, 'learning_rate': 0.00016547190813232432, 'epoch': 1.4}
+
 28%|██▊       | 210/750 [11:17<30:18,  3.37s/it]
 28%|██▊       | 211/750 [11:20<29:10,  3.25s/it]
                                                 
{'loss': 0.2291, 'grad_norm': 0.14019979536533356, 'learning_rate': 0.00016507932891226336, 'epoch': 1.41}
+
 28%|██▊       | 211/750 [11:20<29:10,  3.25s/it]
 28%|██▊       | 212/750 [11:22<26:12,  2.92s/it]
                                                 
{'loss': 0.2653, 'grad_norm': 0.140797421336174, 'learning_rate': 0.00016468953066128386, 'epoch': 1.41}
+
 28%|██▊       | 212/750 [11:22<26:12,  2.92s/it]
 28%|██▊       | 213/750 [11:25<27:14,  3.04s/it]
                                                 
{'loss': 0.2915, 'grad_norm': 0.15229670703411102, 'learning_rate': 0.00016430248070044244, 'epoch': 1.42}
+
 28%|██▊       | 213/750 [11:25<27:14,  3.04s/it]
 29%|██▊       | 214/750 [11:29<27:59,  3.13s/it]
                                                 
{'loss': 0.2745, 'grad_norm': 0.14650197327136993, 'learning_rate': 0.0001639181468858914, 'epoch': 1.43}
+
 29%|██▊       | 214/750 [11:29<27:59,  3.13s/it]
 29%|██▊       | 215/750 [11:32<29:52,  3.35s/it]
                                                 
{'loss': 0.2252, 'grad_norm': 0.13502167165279388, 'learning_rate': 0.00016353649759766664, 'epoch': 1.43}
+
 29%|██▊       | 215/750 [11:32<29:52,  3.35s/it]
 29%|██▉       | 216/750 [11:35<28:18,  3.18s/it]
                                                 
{'loss': 0.2208, 'grad_norm': 0.15106680989265442, 'learning_rate': 0.00016315750172876014, 'epoch': 1.44}
+
 29%|██▉       | 216/750 [11:35<28:18,  3.18s/it]
 29%|██▉       | 217/750 [11:39<28:56,  3.26s/it]
                                                 
{'loss': 0.231, 'grad_norm': 0.1442817598581314, 'learning_rate': 0.00016278112867447063, 'epoch': 1.45}
+
 29%|██▉       | 217/750 [11:39<28:56,  3.26s/it]
 29%|██▉       | 218/750 [11:41<27:25,  3.09s/it]
                                                 
{'loss': 0.2308, 'grad_norm': 0.1436719000339508, 'learning_rate': 0.00016240734832202275, 'epoch': 1.45}
+
 29%|██▉       | 218/750 [11:41<27:25,  3.09s/it]
 29%|██▉       | 219/750 [11:43<24:07,  2.73s/it]
                                                 
{'loss': 0.232, 'grad_norm': 0.1498173624277115, 'learning_rate': 0.00016203613104044751, 'epoch': 1.46}
+
 29%|██▉       | 219/750 [11:43<24:07,  2.73s/it]
 29%|██▉       | 220/750 [11:46<23:36,  2.67s/it]
                                                 
{'loss': 0.2384, 'grad_norm': 0.13557708263397217, 'learning_rate': 0.00016166744767071581, 'epoch': 1.47}
+
 29%|██▉       | 220/750 [11:46<23:36,  2.67s/it]
 29%|██▉       | 221/750 [11:47<20:38,  2.34s/it]
                                                 
{'loss': 0.2621, 'grad_norm': 0.16872936487197876, 'learning_rate': 0.00016130126951611793, 'epoch': 1.47}
+
 29%|██▉       | 221/750 [11:47<20:38,  2.34s/it]
 30%|██▉       | 222/750 [11:50<21:41,  2.46s/it]
                                                 
{'loss': 0.2296, 'grad_norm': 0.1597953736782074, 'learning_rate': 0.0001609375683328815, 'epoch': 1.48}
+
 30%|██▉       | 222/750 [11:50<21:41,  2.46s/it]
 30%|██▉       | 223/750 [11:54<24:36,  2.80s/it]
                                                 
{'loss': 0.2676, 'grad_norm': 0.14711840450763702, 'learning_rate': 0.00016057631632102133, 'epoch': 1.49}
+
 30%|██▉       | 223/750 [11:54<24:36,  2.80s/it]
 30%|██▉       | 224/750 [11:57<26:03,  2.97s/it]
                                                 
{'loss': 0.2296, 'grad_norm': 0.14672479033470154, 'learning_rate': 0.00016021748611541394, 'epoch': 1.49}
+
 30%|██▉       | 224/750 [11:57<26:03,  2.97s/it]
 30%|███       | 225/750 [12:00<25:48,  2.95s/it]
                                                 
{'loss': 0.2569, 'grad_norm': 0.16169887781143188, 'learning_rate': 0.00015986105077709064, 'epoch': 1.5}
+
 30%|███       | 225/750 [12:00<25:48,  2.95s/it]
 30%|███       | 226/750 [12:06<32:33,  3.73s/it]
                                                 
{'loss': 0.2962, 'grad_norm': 0.14002566039562225, 'learning_rate': 0.00015950698378474278, 'epoch': 1.51}
+
 30%|███       | 226/750 [12:06<32:33,  3.73s/it]
 30%|███       | 227/750 [12:08<28:30,  3.27s/it]
                                                 
{'loss': 0.2654, 'grad_norm': 0.14423717558383942, 'learning_rate': 0.00015915525902643283, 'epoch': 1.51}
+
 30%|███       | 227/750 [12:08<28:30,  3.27s/it]
 30%|███       | 228/750 [12:10<26:30,  3.05s/it]
                                                 
{'loss': 0.258, 'grad_norm': 0.14755822718143463, 'learning_rate': 0.0001588058507915059, 'epoch': 1.52}
+
 30%|███       | 228/750 [12:10<26:30,  3.05s/it]
 31%|███       | 229/750 [12:12<23:53,  2.75s/it]
                                                 
{'loss': 0.2313, 'grad_norm': 0.14571203291416168, 'learning_rate': 0.00015845873376269562, 'epoch': 1.53}
+
 31%|███       | 229/750 [12:12<23:53,  2.75s/it]
 31%|███       | 230/750 [12:15<22:36,  2.61s/it]
                                                 
{'loss': 0.2704, 'grad_norm': 0.14023707807064056, 'learning_rate': 0.00015811388300841897, 'epoch': 1.53}
+
 31%|███       | 230/750 [12:15<22:36,  2.61s/it]
 31%|███       | 231/750 [12:18<25:27,  2.94s/it]
                                                 
{'loss': 0.2479, 'grad_norm': 0.13654719293117523, 'learning_rate': 0.00015777127397525472, 'epoch': 1.54}
+
 31%|███       | 231/750 [12:18<25:27,  2.94s/it]
 31%|███       | 232/750 [12:21<25:36,  2.97s/it]
                                                 
{'loss': 0.2431, 'grad_norm': 0.13864630460739136, 'learning_rate': 0.00015743088248060063, 'epoch': 1.55}
+
 31%|███       | 232/750 [12:21<25:36,  2.97s/it]
 31%|███       | 233/750 [12:24<24:25,  2.83s/it]
                                                 
{'loss': 0.2572, 'grad_norm': 0.15521657466888428, 'learning_rate': 0.0001570926847055038, 'epoch': 1.55}
+
 31%|███       | 233/750 [12:24<24:25,  2.83s/it]
 31%|███       | 234/750 [12:26<23:20,  2.71s/it]
                                                 
{'loss': 0.2627, 'grad_norm': 0.16286934912204742, 'learning_rate': 0.00015675665718766006, 'epoch': 1.56}
+
 31%|███       | 234/750 [12:26<23:20,  2.71s/it]
 31%|███▏      | 235/750 [12:29<22:57,  2.68s/it]
                                                 
{'loss': 0.227, 'grad_norm': 0.1435120403766632, 'learning_rate': 0.00015642277681457702, 'epoch': 1.57}
+
 31%|███▏      | 235/750 [12:29<22:57,  2.68s/it]
 31%|███▏      | 236/750 [12:31<21:41,  2.53s/it]
                                                 
{'loss': 0.2616, 'grad_norm': 0.1435043066740036, 'learning_rate': 0.00015609102081689716, 'epoch': 1.57}
+
 31%|███▏      | 236/750 [12:31<21:41,  2.53s/it]
 32%|███▏      | 237/750 [12:33<21:15,  2.49s/it]
                                                 
{'loss': 0.2395, 'grad_norm': 0.1661372035741806, 'learning_rate': 0.00015576136676187527, 'epoch': 1.58}
+
 32%|███▏      | 237/750 [12:33<21:15,  2.49s/it]
 32%|███▏      | 238/750 [12:37<23:28,  2.75s/it]
                                                 
{'loss': 0.2905, 'grad_norm': 0.16668668389320374, 'learning_rate': 0.0001554337925470077, 'epoch': 1.59}
+
 32%|███▏      | 238/750 [12:37<23:28,  2.75s/it]
 32%|███▏      | 239/750 [12:40<25:31,  3.00s/it]
                                                 
{'loss': 0.2264, 'grad_norm': 0.14524315297603607, 'learning_rate': 0.00015510827639380736, 'epoch': 1.59}
+
 32%|███▏      | 239/750 [12:40<25:31,  3.00s/it]
 32%|███▏      | 240/750 [12:44<25:53,  3.05s/it]
                                                 
{'loss': 0.2293, 'grad_norm': 0.1429787576198578, 'learning_rate': 0.00015478479684172258, 'epoch': 1.6}
+
 32%|███▏      | 240/750 [12:44<25:53,  3.05s/it]
 32%|███▏      | 241/750 [12:47<26:08,  3.08s/it]
                                                 
{'loss': 0.2344, 'grad_norm': 0.14680543541908264, 'learning_rate': 0.00015446333274219396, 'epoch': 1.61}
+
 32%|███▏      | 241/750 [12:47<26:08,  3.08s/it]
 32%|███▏      | 242/750 [12:49<24:29,  2.89s/it]
                                                 
{'loss': 0.2629, 'grad_norm': 0.15940262377262115, 'learning_rate': 0.000154143863252847, 'epoch': 1.61}
+
 32%|███▏      | 242/750 [12:49<24:29,  2.89s/it]
 32%|███▏      | 243/750 [12:52<23:55,  2.83s/it]
                                                 
{'loss': 0.239, 'grad_norm': 0.1497008055448532, 'learning_rate': 0.0001538263678318162, 'epoch': 1.62}
+
 32%|███▏      | 243/750 [12:52<23:55,  2.83s/it]
 33%|███▎      | 244/750 [12:54<23:20,  2.77s/it]
                                                 
{'loss': 0.2666, 'grad_norm': 0.17113710939884186, 'learning_rate': 0.00015351082623219707, 'epoch': 1.63}
+
 33%|███▎      | 244/750 [12:54<23:20,  2.77s/it]
 33%|███▎      | 245/750 [12:57<22:21,  2.66s/it]
                                                 
{'loss': 0.2714, 'grad_norm': 0.16685429215431213, 'learning_rate': 0.00015319721849662297, 'epoch': 1.63}
+
 33%|███▎      | 245/750 [12:57<22:21,  2.66s/it]
 33%|███▎      | 246/750 [13:00<22:16,  2.65s/it]
                                                 
{'loss': 0.287, 'grad_norm': 0.1675916165113449, 'learning_rate': 0.0001528855249519633, 'epoch': 1.64}
+
 33%|███▎      | 246/750 [13:00<22:16,  2.65s/it]
 33%|███▎      | 247/750 [13:02<22:12,  2.65s/it]
                                                 
{'loss': 0.2752, 'grad_norm': 0.1514509916305542, 'learning_rate': 0.00015257572620413985, 'epoch': 1.65}
+
 33%|███▎      | 247/750 [13:02<22:12,  2.65s/it]
 33%|███▎      | 248/750 [13:05<23:07,  2.76s/it]
                                                 
{'loss': 0.2482, 'grad_norm': 0.15002918243408203, 'learning_rate': 0.00015226780313305814, 'epoch': 1.65}
+
 33%|███▎      | 248/750 [13:05<23:07,  2.76s/it]
 33%|███▎      | 249/750 [13:07<21:55,  2.63s/it]
                                                 
{'loss': 0.2426, 'grad_norm': 0.14322948455810547, 'learning_rate': 0.00015196173688765093, 'epoch': 1.66}
+
 33%|███▎      | 249/750 [13:07<21:55,  2.63s/it]
 33%|███▎      | 250/750 [13:10<21:53,  2.63s/it]
                                                 
{'loss': 0.2288, 'grad_norm': 0.16433101892471313, 'learning_rate': 0.00015165750888103104, 'epoch': 1.67}
+
 33%|███▎      | 250/750 [13:10<21:53,  2.63s/it]
 33%|███▎      | 251/750 [13:13<21:49,  2.62s/it]
                                                 
{'loss': 0.2466, 'grad_norm': 0.14024463295936584, 'learning_rate': 0.00015135510078575002, 'epoch': 1.67}
+
 33%|███▎      | 251/750 [13:13<21:49,  2.62s/it]
 34%|███▎      | 252/750 [13:16<23:49,  2.87s/it]
                                                 
{'loss': 0.2573, 'grad_norm': 0.15310421586036682, 'learning_rate': 0.00015105449452916094, 'epoch': 1.68}
+
 34%|███▎      | 252/750 [13:16<23:49,  2.87s/it]
 34%|███▎      | 253/750 [13:22<30:21,  3.67s/it]
                                                 
{'loss': 0.2398, 'grad_norm': 0.15072590112686157, 'learning_rate': 0.00015075567228888182, 'epoch': 1.69}
+
 34%|███▎      | 253/750 [13:22<30:21,  3.67s/it]
 34%|███▍      | 254/750 [13:25<28:42,  3.47s/it]
                                                 
{'loss': 0.264, 'grad_norm': 0.15282756090164185, 'learning_rate': 0.0001504586164883572, 'epoch': 1.69}
+
 34%|███▍      | 254/750 [13:25<28:42,  3.47s/it]
 34%|███▍      | 255/750 [13:27<26:19,  3.19s/it]
                                                 
{'loss': 0.2562, 'grad_norm': 0.14381791651248932, 'learning_rate': 0.00015016330979251603, 'epoch': 1.7}
+
 34%|███▍      | 255/750 [13:27<26:19,  3.19s/it]
 34%|███▍      | 256/750 [13:30<24:17,  2.95s/it]
                                                 
{'loss': 0.2594, 'grad_norm': 0.16199229657649994, 'learning_rate': 0.00014986973510352247, 'epoch': 1.71}
+
 34%|███▍      | 256/750 [13:30<24:17,  2.95s/it]
 34%|███▍      | 257/750 [13:32<23:56,  2.91s/it]
                                                 
{'loss': 0.2566, 'grad_norm': 0.1585993617773056, 'learning_rate': 0.0001495778755566181, 'epoch': 1.71}
+
 34%|███▍      | 257/750 [13:32<23:56,  2.91s/it]
 34%|███▍      | 258/750 [13:34<21:32,  2.63s/it]
                                                 
{'loss': 0.2315, 'grad_norm': 0.15047159790992737, 'learning_rate': 0.0001492877145160524, 'epoch': 1.72}
+
 34%|███▍      | 258/750 [13:34<21:32,  2.63s/it]
 35%|███▍      | 259/750 [13:36<19:41,  2.41s/it]
                                                 
{'loss': 0.2758, 'grad_norm': 0.14615359902381897, 'learning_rate': 0.00014899923557110017, 'epoch': 1.73}
+
 35%|███▍      | 259/750 [13:36<19:41,  2.41s/it]
 35%|███▍      | 260/750 [13:38<18:54,  2.32s/it]
                                                 
{'loss': 0.2739, 'grad_norm': 0.15001897513866425, 'learning_rate': 0.0001487124225321631, 'epoch': 1.73}
+
 35%|███▍      | 260/750 [13:38<18:54,  2.32s/it]
 35%|███▍      | 261/750 [13:40<17:55,  2.20s/it]
                                                 
{'loss': 0.2709, 'grad_norm': 0.16185350716114044, 'learning_rate': 0.00014842725942695355, 'epoch': 1.74}
+
 35%|███▍      | 261/750 [13:40<17:55,  2.20s/it]
 35%|███▍      | 262/750 [13:43<19:43,  2.42s/it]
                                                 
{'loss': 0.2102, 'grad_norm': 0.1416780799627304, 'learning_rate': 0.0001481437304967584, 'epoch': 1.75}
+
 35%|███▍      | 262/750 [13:43<19:43,  2.42s/it]
 35%|███▌      | 263/750 [13:47<23:35,  2.91s/it]
                                                 
{'loss': 0.2569, 'grad_norm': 0.15291401743888855, 'learning_rate': 0.00014786182019278145, 'epoch': 1.75}
+
 35%|███▌      | 263/750 [13:47<23:35,  2.91s/it]
 35%|███▌      | 264/750 [13:49<21:31,  2.66s/it]
                                                 
{'loss': 0.2387, 'grad_norm': 0.14845313131809235, 'learning_rate': 0.0001475815131725618, 'epoch': 1.76}
+
 35%|███▌      | 264/750 [13:49<21:31,  2.66s/it]
 35%|███▌      | 265/750 [13:52<21:53,  2.71s/it]
                                                 
{'loss': 0.2653, 'grad_norm': 0.14375221729278564, 'learning_rate': 0.00014730279429646666, 'epoch': 1.77}
+
 35%|███▌      | 265/750 [13:52<21:53,  2.71s/it]
 35%|███▌      | 266/750 [13:55<22:13,  2.75s/it]
                                                 
{'loss': 0.2439, 'grad_norm': 0.15388962626457214, 'learning_rate': 0.0001470256486242569, 'epoch': 1.77}
+
 35%|███▌      | 266/750 [13:55<22:13,  2.75s/it]
 36%|███▌      | 267/750 [13:58<22:08,  2.75s/it]
                                                 
{'loss': 0.2338, 'grad_norm': 0.15162217617034912, 'learning_rate': 0.00014675006141172335, 'epoch': 1.78}
+
 36%|███▌      | 267/750 [13:58<22:08,  2.75s/it]
 36%|███▌      | 268/750 [14:01<22:48,  2.84s/it]
                                                 
{'loss': 0.2619, 'grad_norm': 0.1475297510623932, 'learning_rate': 0.00014647601810739195, 'epoch': 1.79}
+
 36%|███▌      | 268/750 [14:01<22:48,  2.84s/it]
 36%|███▌      | 269/750 [14:03<21:32,  2.69s/it]
                                                 
{'loss': 0.3039, 'grad_norm': 0.16632747650146484, 'learning_rate': 0.0001462035043492969, 'epoch': 1.79}
+
 36%|███▌      | 269/750 [14:03<21:32,  2.69s/it]
 36%|███▌      | 270/750 [14:09<28:42,  3.59s/it]
                                                 
{'loss': 0.2451, 'grad_norm': 0.14030596613883972, 'learning_rate': 0.00014593250596181887, 'epoch': 1.8}
+
 36%|███▌      | 270/750 [14:09<28:42,  3.59s/it]
 36%|███▌      | 271/750 [14:12<26:33,  3.33s/it]
                                                 
{'loss': 0.2417, 'grad_norm': 0.14720116555690765, 'learning_rate': 0.00014566300895258813, 'epoch': 1.81}
+
 36%|███▌      | 271/750 [14:12<26:33,  3.33s/it]
 36%|███▋      | 272/750 [14:17<32:05,  4.03s/it]
                                                 
{'loss': 0.2244, 'grad_norm': 0.15774893760681152, 'learning_rate': 0.00014539499950944992, 'epoch': 1.81}
+
 36%|███▋      | 272/750 [14:17<32:05,  4.03s/it]
 36%|███▋      | 273/750 [14:19<27:00,  3.40s/it]
                                                 
{'loss': 0.2352, 'grad_norm': 0.15491846203804016, 'learning_rate': 0.00014512846399749107, 'epoch': 1.82}
+
 36%|███▋      | 273/750 [14:19<27:00,  3.40s/it]
 37%|███▋      | 274/750 [14:22<26:16,  3.31s/it]
                                                 
{'loss': 0.2007, 'grad_norm': 0.13752566277980804, 'learning_rate': 0.00014486338895612657, 'epoch': 1.83}
+
 37%|███▋      | 274/750 [14:22<26:16,  3.31s/it]
 37%|███▋      | 275/750 [14:24<23:13,  2.93s/it]
                                                 
{'loss': 0.2242, 'grad_norm': 0.15706242620944977, 'learning_rate': 0.00014459976109624422, 'epoch': 1.83}
+
 37%|███▋      | 275/750 [14:24<23:13,  2.93s/it]
 37%|███▋      | 276/750 [14:27<22:57,  2.91s/it]
                                                 
{'loss': 0.2498, 'grad_norm': 0.15001772344112396, 'learning_rate': 0.00014433756729740648, 'epoch': 1.84}
+
 37%|███▋      | 276/750 [14:27<22:57,  2.91s/it]
 37%|███▋      | 277/750 [14:29<20:57,  2.66s/it]
                                                 
{'loss': 0.3114, 'grad_norm': 0.19167815148830414, 'learning_rate': 0.00014407679460510741, 'epoch': 1.85}
+
 37%|███▋      | 277/750 [14:29<20:57,  2.66s/it]
 37%|███▋      | 278/750 [14:32<20:13,  2.57s/it]
                                                 
{'loss': 0.2356, 'grad_norm': 0.16021162271499634, 'learning_rate': 0.00014381743022808492, 'epoch': 1.85}
+
 37%|███▋      | 278/750 [14:32<20:13,  2.57s/it]
 37%|███▋      | 279/750 [14:34<20:11,  2.57s/it]
                                                 
{'loss': 0.2334, 'grad_norm': 0.1472308337688446, 'learning_rate': 0.00014355946153568483, 'epoch': 1.86}
+
 37%|███▋      | 279/750 [14:34<20:11,  2.57s/it]
 37%|███▋      | 280/750 [14:37<20:53,  2.67s/it]
                                                 
{'loss': 0.2646, 'grad_norm': 0.15674757957458496, 'learning_rate': 0.0001433028760552777, 'epoch': 1.87}
+
 37%|███▋      | 280/750 [14:37<20:53,  2.67s/it]
 37%|███▋      | 281/750 [14:40<21:18,  2.73s/it]
                                                 
{'loss': 0.2907, 'grad_norm': 0.1543518751859665, 'learning_rate': 0.00014304766146972588, 'epoch': 1.87}
+
 37%|███▋      | 281/750 [14:40<21:18,  2.73s/it]
 38%|███▊      | 282/750 [14:43<22:05,  2.83s/it]
                                                 
{'loss': 0.263, 'grad_norm': 0.15293796360492706, 'learning_rate': 0.0001427938056148999, 'epoch': 1.88}
+
 38%|███▊      | 282/750 [14:43<22:05,  2.83s/it]
 38%|███▊      | 283/750 [14:46<21:39,  2.78s/it]
                                                 
{'loss': 0.2359, 'grad_norm': 0.14394539594650269, 'learning_rate': 0.00014254129647724352, 'epoch': 1.89}
+
 38%|███▊      | 283/750 [14:46<21:39,  2.78s/it]
 38%|███▊      | 284/750 [14:49<21:49,  2.81s/it]
                                                 
{'loss': 0.2561, 'grad_norm': 0.14988663792610168, 'learning_rate': 0.00014229012219138556, 'epoch': 1.89}
+
 38%|███▊      | 284/750 [14:49<21:49,  2.81s/it]
 38%|███▊      | 285/750 [14:51<19:57,  2.58s/it]
                                                 
{'loss': 0.2823, 'grad_norm': 0.13968853652477264, 'learning_rate': 0.00014204027103779837, 'epoch': 1.9}
+
 38%|███▊      | 285/750 [14:51<19:57,  2.58s/it]
 38%|███▊      | 286/750 [14:54<20:34,  2.66s/it]
                                                 
{'loss': 0.2608, 'grad_norm': 0.143513485789299, 'learning_rate': 0.0001417917314405008, 'epoch': 1.91}
+
 38%|███▊      | 286/750 [14:54<20:34,  2.66s/it]
 38%|███▊      | 287/750 [14:57<21:21,  2.77s/it]
                                                 
{'loss': 0.2654, 'grad_norm': 0.15734688937664032, 'learning_rate': 0.0001415444919648056, 'epoch': 1.91}
+
 38%|███▊      | 287/750 [14:57<21:21,  2.77s/it]
 38%|███▊      | 288/750 [15:00<22:23,  2.91s/it]
                                                 
{'loss': 0.2484, 'grad_norm': 0.14549553394317627, 'learning_rate': 0.00014129854131510975, 'epoch': 1.92}
+
 38%|███▊      | 288/750 [15:00<22:23,  2.91s/it]
 39%|███▊      | 289/750 [15:02<21:56,  2.86s/it]
                                                 
{'loss': 0.2257, 'grad_norm': 0.13937290012836456, 'learning_rate': 0.00014105386833272704, 'epoch': 1.93}
+
 39%|███▊      | 289/750 [15:02<21:56,  2.86s/it]
 39%|███▊      | 290/750 [15:06<22:45,  2.97s/it]
                                                 
{'loss': 0.2609, 'grad_norm': 0.1811768114566803, 'learning_rate': 0.0001408104619937615, 'epoch': 1.93}
+
 39%|███▊      | 290/750 [15:06<22:45,  2.97s/it]
 39%|███▉      | 291/750 [15:08<21:45,  2.84s/it]
                                                 
{'loss': 0.2317, 'grad_norm': 0.1553860902786255, 'learning_rate': 0.00014056831140702128, 'epoch': 1.94}
+
 39%|███▉      | 291/750 [15:08<21:45,  2.84s/it]
 39%|███▉      | 292/750 [15:12<22:50,  2.99s/it]
                                                 
{'loss': 0.2574, 'grad_norm': 0.1608595997095108, 'learning_rate': 0.0001403274058119718, 'epoch': 1.95}
+
 39%|███▉      | 292/750 [15:12<22:50,  2.99s/it]
 39%|███▉      | 293/750 [15:14<21:47,  2.86s/it]
                                                 
{'loss': 0.29, 'grad_norm': 0.16590401530265808, 'learning_rate': 0.00014008773457672727, 'epoch': 1.95}
+
 39%|███▉      | 293/750 [15:14<21:47,  2.86s/it]
 39%|███▉      | 294/750 [15:16<19:35,  2.58s/it]
                                                 
{'loss': 0.2286, 'grad_norm': 0.1594795286655426, 'learning_rate': 0.00013984928719608012, 'epoch': 1.96}
+
 39%|███▉      | 294/750 [15:16<19:35,  2.58s/it]
 39%|███▉      | 295/750 [15:20<23:18,  3.07s/it]
                                                 
{'loss': 0.2489, 'grad_norm': 0.16078892350196838, 'learning_rate': 0.0001396120532895667, 'epoch': 1.97}
+
 39%|███▉      | 295/750 [15:20<23:18,  3.07s/it]
 39%|███▉      | 296/750 [15:23<22:21,  2.95s/it]
                                                 
{'loss': 0.2979, 'grad_norm': 0.17581714689731598, 'learning_rate': 0.00013937602259956938, 'epoch': 1.97}
+
 39%|███▉      | 296/750 [15:23<22:21,  2.95s/it]
 40%|███▉      | 297/750 [15:26<21:31,  2.85s/it]
                                                 
{'loss': 0.2471, 'grad_norm': 0.14835497736930847, 'learning_rate': 0.00013914118498945362, 'epoch': 1.98}
+
 40%|███▉      | 297/750 [15:26<21:31,  2.85s/it]
 40%|███▉      | 298/750 [15:28<21:25,  2.84s/it]
                                                 
{'loss': 0.2198, 'grad_norm': 0.1524120569229126, 'learning_rate': 0.00013890753044173962, 'epoch': 1.99}
+
 40%|███▉      | 298/750 [15:28<21:25,  2.84s/it]
 40%|███▉      | 299/750 [15:33<24:41,  3.28s/it]
                                                 
{'loss': 0.2373, 'grad_norm': 0.15511971712112427, 'learning_rate': 0.0001386750490563073, 'epoch': 1.99}
+
 40%|███▉      | 299/750 [15:33<24:41,  3.28s/it]
 40%|████      | 300/750 [15:37<27:47,  3.71s/it]
                                                 
{'loss': 0.2086, 'grad_norm': 0.14986000955104828, 'learning_rate': 0.00013844373104863458, 'epoch': 2.0}
+
 40%|████      | 300/750 [15:37<27:47,  3.71s/it]
 40%|████      | 301/750 [15:49<45:03,  6.02s/it]
                                                 
{'loss': 0.2123, 'grad_norm': 0.16342109441757202, 'learning_rate': 0.00013821356674806781, 'epoch': 2.01}
+
 40%|████      | 301/750 [15:49<45:03,  6.02s/it]
 40%|████      | 302/750 [15:51<36:09,  4.84s/it]
                                                 
{'loss': 0.2274, 'grad_norm': 0.16324245929718018, 'learning_rate': 0.00013798454659612338, 'epoch': 2.01}
+
 40%|████      | 302/750 [15:51<36:09,  4.84s/it]
 40%|████      | 303/750 [15:54<32:29,  4.36s/it]
                                                 
{'loss': 0.1719, 'grad_norm': 0.14752919971942902, 'learning_rate': 0.00013775666114482078, 'epoch': 2.02}
+
 40%|████      | 303/750 [15:54<32:29,  4.36s/it]
 41%|████      | 304/750 [15:57<29:07,  3.92s/it]
                                                 
{'loss': 0.2069, 'grad_norm': 0.1696600764989853, 'learning_rate': 0.0001375299010550452, 'epoch': 2.03}
+
 41%|████      | 304/750 [15:57<29:07,  3.92s/it]
 41%|████      | 305/750 [15:59<25:37,  3.46s/it]
                                                 
{'loss': 0.1961, 'grad_norm': 0.15505684912204742, 'learning_rate': 0.0001373042570949402, 'epoch': 2.03}
+
 41%|████      | 305/750 [15:59<25:37,  3.46s/it]
 41%|████      | 306/750 [16:03<24:48,  3.35s/it]
                                                 
{'loss': 0.1906, 'grad_norm': 0.14594782888889313, 'learning_rate': 0.00013707972013832906, 'epoch': 2.04}
+
 41%|████      | 306/750 [16:03<24:48,  3.35s/it]
 41%|████      | 307/750 [16:05<22:14,  3.01s/it]
                                                 
{'loss': 0.2124, 'grad_norm': 0.15274815261363983, 'learning_rate': 0.0001368562811631643, 'epoch': 2.05}
+
 41%|████      | 307/750 [16:05<22:14,  3.01s/it]
 41%|████      | 308/750 [16:09<24:23,  3.31s/it]
                                                 
{'loss': 0.1701, 'grad_norm': 0.1536024510860443, 'learning_rate': 0.0001366339312500053, 'epoch': 2.05}
+
 41%|████      | 308/750 [16:09<24:23,  3.31s/it]
 41%|████      | 309/750 [16:11<22:03,  3.00s/it]
                                                 
{'loss': 0.2438, 'grad_norm': 0.2351594716310501, 'learning_rate': 0.00013641266158052266, 'epoch': 2.06}
+
 41%|████      | 309/750 [16:11<22:03,  3.00s/it]
 41%|████▏     | 310/750 [16:14<20:57,  2.86s/it]
                                                 
{'loss': 0.2276, 'grad_norm': 0.21760991215705872, 'learning_rate': 0.00013619246343602937, 'epoch': 2.07}
+
 41%|████▏     | 310/750 [16:14<20:57,  2.86s/it]
 41%|████▏     | 311/750 [16:16<19:36,  2.68s/it]
                                                 
{'loss': 0.2232, 'grad_norm': 0.19298413395881653, 'learning_rate': 0.00013597332819603774, 'epoch': 2.07}
+
 41%|████▏     | 311/750 [16:16<19:36,  2.68s/it]
 42%|████▏     | 312/750 [16:21<25:50,  3.54s/it]
                                                 
{'loss': 0.2117, 'grad_norm': 0.20474325120449066, 'learning_rate': 0.00013575524733684213, 'epoch': 2.08}
+
 42%|████▏     | 312/750 [16:21<25:50,  3.54s/it]
 42%|████▏     | 313/750 [16:24<23:18,  3.20s/it]
                                                 
{'loss': 0.1832, 'grad_norm': 0.16953983902931213, 'learning_rate': 0.00013553821243012613, 'epoch': 2.09}
+
 42%|████▏     | 313/750 [16:24<23:18,  3.20s/it]
 42%|████▏     | 314/750 [16:26<21:54,  3.01s/it]
                                                 
{'loss': 0.1881, 'grad_norm': 0.1634465456008911, 'learning_rate': 0.00013532221514159462, 'epoch': 2.09}
+
 42%|████▏     | 314/750 [16:26<21:54,  3.01s/it]
 42%|████▏     | 315/750 [16:29<20:49,  2.87s/it]
                                                 
{'loss': 0.2179, 'grad_norm': 0.16336411237716675, 'learning_rate': 0.00013510724722962957, 'epoch': 2.1}
+
 42%|████▏     | 315/750 [16:29<20:49,  2.87s/it]
 42%|████▏     | 316/750 [16:32<20:50,  2.88s/it]
                                                 
{'loss': 0.2227, 'grad_norm': 0.1706719547510147, 'learning_rate': 0.00013489330054396908, 'epoch': 2.11}
+
 42%|████▏     | 316/750 [16:32<20:50,  2.88s/it]
 42%|████▏     | 317/750 [16:35<20:48,  2.88s/it]
                                                 
{'loss': 0.2224, 'grad_norm': 0.17670485377311707, 'learning_rate': 0.0001346803670244095, 'epoch': 2.11}
+
 42%|████▏     | 317/750 [16:35<20:48,  2.88s/it]
 42%|████▏     | 318/750 [16:38<21:31,  2.99s/it]
                                                 
{'loss': 0.1885, 'grad_norm': 0.1563681960105896, 'learning_rate': 0.00013446843869953018, 'epoch': 2.12}
+
 42%|████▏     | 318/750 [16:38<21:31,  2.99s/it]
 43%|████▎     | 319/750 [16:40<19:47,  2.76s/it]
                                                 
{'loss': 0.1736, 'grad_norm': 0.1708376407623291, 'learning_rate': 0.0001342575076854399, 'epoch': 2.13}
+
 43%|████▎     | 319/750 [16:40<19:47,  2.76s/it]
 43%|████▎     | 320/750 [16:43<18:52,  2.63s/it]
                                                 
{'loss': 0.1953, 'grad_norm': 0.17504575848579407, 'learning_rate': 0.00013404756618454512, 'epoch': 2.13}
+
 43%|████▎     | 320/750 [16:43<18:52,  2.63s/it]
 43%|████▎     | 321/750 [16:44<17:19,  2.42s/it]
                                                 
{'loss': 0.2121, 'grad_norm': 0.18623320758342743, 'learning_rate': 0.00013383860648433912, 'epoch': 2.14}
+
 43%|████▎     | 321/750 [16:44<17:19,  2.42s/it]
 43%|████▎     | 322/750 [16:48<20:40,  2.90s/it]
                                                 
{'loss': 0.1927, 'grad_norm': 0.1664789468050003, 'learning_rate': 0.0001336306209562122, 'epoch': 2.15}
+
 43%|████▎     | 322/750 [16:48<20:40,  2.90s/it]
 43%|████▎     | 323/750 [16:51<20:10,  2.83s/it]
                                                 
{'loss': 0.2084, 'grad_norm': 0.19429877400398254, 'learning_rate': 0.0001334236020542817, 'epoch': 2.15}
+
 43%|████▎     | 323/750 [16:51<20:10,  2.83s/it]
 43%|████▎     | 324/750 [16:54<20:30,  2.89s/it]
                                                 
{'loss': 0.2393, 'grad_norm': 0.19420652091503143, 'learning_rate': 0.00013321754231424221, 'epoch': 2.16}
+
 43%|████▎     | 324/750 [16:54<20:30,  2.89s/it]
 43%|████▎     | 325/750 [16:57<20:20,  2.87s/it]
                                                 
{'loss': 0.2162, 'grad_norm': 0.195955291390419, 'learning_rate': 0.00013301243435223526, 'epoch': 2.17}
+
 43%|████▎     | 325/750 [16:57<20:20,  2.87s/it]
 43%|████▎     | 326/750 [16:59<19:02,  2.69s/it]
                                                 
{'loss': 0.2058, 'grad_norm': 0.178892120718956, 'learning_rate': 0.00013280827086373782, 'epoch': 2.17}
+
 43%|████▎     | 326/750 [16:59<19:02,  2.69s/it]
 44%|████▎     | 327/750 [17:01<16:53,  2.40s/it]
                                                 
{'loss': 0.2263, 'grad_norm': 0.21568411588668823, 'learning_rate': 0.00013260504462246983, 'epoch': 2.18}
+
 44%|████▎     | 327/750 [17:01<16:53,  2.40s/it]
 44%|████▎     | 328/750 [17:06<21:36,  3.07s/it]
                                                 
{'loss': 0.1578, 'grad_norm': 0.16020487248897552, 'learning_rate': 0.00013240274847931992, 'epoch': 2.19}
+
 44%|████▎     | 328/750 [17:06<21:36,  3.07s/it]
 44%|████▍     | 329/750 [17:08<19:58,  2.85s/it]
                                                 
{'loss': 0.2157, 'grad_norm': 0.1962222456932068, 'learning_rate': 0.00013220137536128893, 'epoch': 2.19}
+
 44%|████▍     | 329/750 [17:08<19:58,  2.85s/it]
 44%|████▍     | 330/750 [17:11<20:00,  2.86s/it]
                                                 
{'loss': 0.1686, 'grad_norm': 0.18296131491661072, 'learning_rate': 0.00013200091827045155, 'epoch': 2.2}
+
 44%|████▍     | 330/750 [17:11<20:00,  2.86s/it]
 44%|████▍     | 331/750 [17:15<22:06,  3.17s/it]
                                                 
{'loss': 0.2083, 'grad_norm': 0.18830646574497223, 'learning_rate': 0.00013180137028293452, 'epoch': 2.21}
+
 44%|████▍     | 331/750 [17:15<22:06,  3.17s/it]
 44%|████▍     | 332/750 [17:16<18:59,  2.73s/it]
                                                 
{'loss': 0.2125, 'grad_norm': 0.20384907722473145, 'learning_rate': 0.0001316027245479125, 'epoch': 2.21}
+
 44%|████▍     | 332/750 [17:16<18:59,  2.73s/it]
 44%|████▍     | 333/750 [17:19<18:09,  2.61s/it]
                                                 
{'loss': 0.1924, 'grad_norm': 0.19217239320278168, 'learning_rate': 0.00013140497428662003, 'epoch': 2.22}
+
 44%|████▍     | 333/750 [17:19<18:09,  2.61s/it]
 45%|████▍     | 334/750 [17:21<18:16,  2.64s/it]
                                                 
{'loss': 0.2674, 'grad_norm': 0.2167179435491562, 'learning_rate': 0.00013120811279137981, 'epoch': 2.23}
+
 45%|████▍     | 334/750 [17:21<18:16,  2.64s/it]
 45%|████▍     | 335/750 [17:24<18:40,  2.70s/it]
                                                 
{'loss': 0.1801, 'grad_norm': 0.17744633555412292, 'learning_rate': 0.0001310121334246474, 'epoch': 2.23}
+
 45%|████▍     | 335/750 [17:24<18:40,  2.70s/it]
 45%|████▍     | 336/750 [17:27<19:02,  2.76s/it]
                                                 
{'loss': 0.1591, 'grad_norm': 0.16290077567100525, 'learning_rate': 0.0001308170296180709, 'epoch': 2.24}
+
 45%|████▍     | 336/750 [17:27<19:02,  2.76s/it]
 45%|████▍     | 337/750 [17:30<18:49,  2.73s/it]
                                                 
{'loss': 0.2132, 'grad_norm': 0.1816015988588333, 'learning_rate': 0.00013062279487156654, 'epoch': 2.25}
+
 45%|████▍     | 337/750 [17:30<18:49,  2.73s/it]
 45%|████▌     | 338/750 [17:32<18:30,  2.70s/it]
                                                 
{'loss': 0.2133, 'grad_norm': 0.2061437964439392, 'learning_rate': 0.00013042942275240903, 'epoch': 2.25}
+
 45%|████▌     | 338/750 [17:32<18:30,  2.70s/it]
 45%|████▌     | 339/750 [17:35<18:14,  2.66s/it]
                                                 
{'loss': 0.2037, 'grad_norm': 0.1909235715866089, 'learning_rate': 0.0001302369068943367, 'epoch': 2.26}
+
 45%|████▌     | 339/750 [17:35<18:14,  2.66s/it]
 45%|████▌     | 340/750 [17:38<19:25,  2.84s/it]
                                                 
{'loss': 0.1799, 'grad_norm': 0.17861449718475342, 'learning_rate': 0.00013004524099667142, 'epoch': 2.27}
+
 45%|████▌     | 340/750 [17:38<19:25,  2.84s/it]
 45%|████▌     | 341/750 [17:41<19:43,  2.89s/it]
                                                 
{'loss': 0.1985, 'grad_norm': 0.17117547988891602, 'learning_rate': 0.00012985441882345223, 'epoch': 2.27}
+
 45%|████▌     | 341/750 [17:41<19:43,  2.89s/it]
 46%|████▌     | 342/750 [17:44<19:06,  2.81s/it]
                                                 
{'loss': 0.2045, 'grad_norm': 0.17320069670677185, 'learning_rate': 0.00012966443420258316, 'epoch': 2.28}
+
 46%|████▌     | 342/750 [17:44<19:06,  2.81s/it]
 46%|████▌     | 343/750 [17:48<20:37,  3.04s/it]
                                                 
{'loss': 0.1978, 'grad_norm': 0.17472076416015625, 'learning_rate': 0.00012947528102499512, 'epoch': 2.29}
+
 46%|████▌     | 343/750 [17:48<20:37,  3.04s/it]
 46%|████▌     | 344/750 [17:50<19:02,  2.81s/it]
                                                 
{'loss': 0.1973, 'grad_norm': 0.1752399057149887, 'learning_rate': 0.00012928695324382025, 'epoch': 2.29}
+
 46%|████▌     | 344/750 [17:50<19:02,  2.81s/it]
 46%|████▌     | 345/750 [17:52<17:25,  2.58s/it]
                                                 
{'loss': 0.2135, 'grad_norm': 0.20518158376216888, 'learning_rate': 0.00012909944487358055, 'epoch': 2.3}
+
 46%|████▌     | 345/750 [17:52<17:25,  2.58s/it]
 46%|████▌     | 346/750 [17:54<17:17,  2.57s/it]
                                                 
{'loss': 0.2126, 'grad_norm': 0.20062129199504852, 'learning_rate': 0.0001289127499893885, 'epoch': 2.31}
+
 46%|████▌     | 346/750 [17:54<17:17,  2.57s/it]
 46%|████▋     | 347/750 [17:58<19:10,  2.85s/it]
                                                 
{'loss': 0.1789, 'grad_norm': 0.16289368271827698, 'learning_rate': 0.00012872686272616092, 'epoch': 2.31}
+
 46%|████▋     | 347/750 [17:58<19:10,  2.85s/it]
 46%|████▋     | 348/750 [18:00<18:33,  2.77s/it]
                                                 
{'loss': 0.208, 'grad_norm': 0.1817348152399063, 'learning_rate': 0.0001285417772778451, 'epoch': 2.32}
+
 46%|████▋     | 348/750 [18:00<18:33,  2.77s/it]
 47%|████▋     | 349/750 [18:03<17:40,  2.64s/it]
                                                 
{'loss': 0.2021, 'grad_norm': 0.19883078336715698, 'learning_rate': 0.0001283574878966568, 'epoch': 2.33}
+
 47%|████▋     | 349/750 [18:03<17:40,  2.64s/it]
 47%|████▋     | 350/750 [18:06<19:28,  2.92s/it]
                                                 
{'loss': 0.204, 'grad_norm': 0.18379899859428406, 'learning_rate': 0.00012817398889233115, 'epoch': 2.33}
+
 47%|████▋     | 350/750 [18:06<19:28,  2.92s/it]
 47%|████▋     | 351/750 [18:09<17:53,  2.69s/it]
                                                 
{'loss': 0.1914, 'grad_norm': 0.19058142602443695, 'learning_rate': 0.00012799127463138408, 'epoch': 2.34}
+
 47%|████▋     | 351/750 [18:09<17:53,  2.69s/it]
 47%|████▋     | 352/750 [18:12<18:54,  2.85s/it]
                                                 
{'loss': 0.1966, 'grad_norm': 0.1994251161813736, 'learning_rate': 0.00012780933953638628, 'epoch': 2.35}
+
 47%|████▋     | 352/750 [18:12<18:54,  2.85s/it]
 47%|████▋     | 353/750 [18:15<19:31,  2.95s/it]
                                                 
{'loss': 0.204, 'grad_norm': 0.2002660185098648, 'learning_rate': 0.00012762817808524798, 'epoch': 2.35}
+
 47%|████▋     | 353/750 [18:15<19:31,  2.95s/it]
 47%|████▋     | 354/750 [18:18<19:49,  3.00s/it]
                                                 
{'loss': 0.2281, 'grad_norm': 0.212173193693161, 'learning_rate': 0.00012744778481051504, 'epoch': 2.36}
+
 47%|████▋     | 354/750 [18:18<19:49,  3.00s/it]
 47%|████▋     | 355/750 [18:20<17:41,  2.69s/it]
                                                 
{'loss': 0.1838, 'grad_norm': 0.18124210834503174, 'learning_rate': 0.0001272681542986758, 'epoch': 2.37}
+
 47%|████▋     | 355/750 [18:20<17:41,  2.69s/it]
 47%|████▋     | 356/750 [18:23<18:01,  2.74s/it]
                                                 
{'loss': 0.2216, 'grad_norm': 0.2050614356994629, 'learning_rate': 0.0001270892811894789, 'epoch': 2.37}
+
 47%|████▋     | 356/750 [18:23<18:01,  2.74s/it]
 48%|████▊     | 357/750 [18:26<18:09,  2.77s/it]
                                                 
{'loss': 0.2047, 'grad_norm': 0.1962577849626541, 'learning_rate': 0.00012691116017526124, 'epoch': 2.38}
+
 48%|████▊     | 357/750 [18:26<18:09,  2.77s/it]
 48%|████▊     | 358/750 [18:29<18:24,  2.82s/it]
                                                 
{'loss': 0.225, 'grad_norm': 0.20875343680381775, 'learning_rate': 0.00012673378600028677, 'epoch': 2.39}
+
 48%|████▊     | 358/750 [18:29<18:24,  2.82s/it]
 48%|████▊     | 359/750 [18:31<17:39,  2.71s/it]
                                                 
{'loss': 0.2648, 'grad_norm': 0.2457055151462555, 'learning_rate': 0.00012655715346009488, 'epoch': 2.39}
+
 48%|████▊     | 359/750 [18:31<17:39,  2.71s/it]
 48%|████▊     | 360/750 [18:34<18:20,  2.82s/it]
                                                 
{'loss': 0.2421, 'grad_norm': 0.20507971942424774, 'learning_rate': 0.00012638125740085918, 'epoch': 2.4}
+
 48%|████▊     | 360/750 [18:34<18:20,  2.82s/it]
 48%|████▊     | 361/750 [18:37<18:20,  2.83s/it]
                                                 
{'loss': 0.2151, 'grad_norm': 0.21578699350357056, 'learning_rate': 0.0001262060927187558, 'epoch': 2.41}
+
 48%|████▊     | 361/750 [18:37<18:20,  2.83s/it]
 48%|████▊     | 362/750 [18:39<17:15,  2.67s/it]
                                                 
{'loss': 0.1879, 'grad_norm': 0.21760186553001404, 'learning_rate': 0.00012603165435934137, 'epoch': 2.41}
+
 48%|████▊     | 362/750 [18:39<17:15,  2.67s/it]
 48%|████▊     | 363/750 [18:42<16:51,  2.61s/it]
                                                 
{'loss': 0.1789, 'grad_norm': 0.20178544521331787, 'learning_rate': 0.00012585793731694053, 'epoch': 2.42}
+
 48%|████▊     | 363/750 [18:42<16:51,  2.61s/it]
 49%|████▊     | 364/750 [18:45<18:43,  2.91s/it]
                                                 
{'loss': 0.2216, 'grad_norm': 0.2194928675889969, 'learning_rate': 0.00012568493663404258, 'epoch': 2.43}
+
 49%|████▊     | 364/750 [18:45<18:43,  2.91s/it]
 49%|████▊     | 365/750 [18:48<17:42,  2.76s/it]
                                                 
{'loss': 0.1964, 'grad_norm': 0.19872188568115234, 'learning_rate': 0.00012551264740070717, 'epoch': 2.43}
+
 49%|████▊     | 365/750 [18:48<17:42,  2.76s/it]
 49%|████▉     | 366/750 [18:50<17:10,  2.68s/it]
                                                 
{'loss': 0.2615, 'grad_norm': 0.2289508730173111, 'learning_rate': 0.0001253410647539792, 'epoch': 2.44}
+
 49%|████▉     | 366/750 [18:50<17:10,  2.68s/it]
 49%|████▉     | 367/750 [18:53<16:15,  2.55s/it]
                                                 
{'loss': 0.1688, 'grad_norm': 0.1865498423576355, 'learning_rate': 0.00012517018387731205, 'epoch': 2.45}
+
 49%|████▉     | 367/750 [18:53<16:15,  2.55s/it]
 49%|████▉     | 368/750 [18:55<16:20,  2.57s/it]
                                                 
{'loss': 0.2255, 'grad_norm': 0.20943808555603027, 'learning_rate': 0.000125, 'epoch': 2.45}
+
 49%|████▉     | 368/750 [18:55<16:20,  2.57s/it]
 49%|████▉     | 369/750 [18:57<15:43,  2.48s/it]
                                                 
{'loss': 0.1923, 'grad_norm': 0.19173435866832733, 'learning_rate': 0.0001248305083966186, 'epoch': 2.46}
+
 49%|████▉     | 369/750 [18:57<15:43,  2.48s/it]
 49%|████▉     | 370/750 [19:00<15:16,  2.41s/it]
                                                 
{'loss': 0.2127, 'grad_norm': 0.21553656458854675, 'learning_rate': 0.00012466170438647366, 'epoch': 2.47}
+
 49%|████▉     | 370/750 [19:00<15:16,  2.41s/it]
 49%|████▉     | 371/750 [19:04<19:07,  3.03s/it]
                                                 
{'loss': 0.2079, 'grad_norm': 0.20172956585884094, 'learning_rate': 0.00012449358333305828, 'epoch': 2.47}
+
 49%|████▉     | 371/750 [19:04<19:07,  3.03s/it]
 50%|████▉     | 372/750 [19:08<20:22,  3.23s/it]
                                                 
{'loss': 0.226, 'grad_norm': 0.20866543054580688, 'learning_rate': 0.00012432614064351803, 'epoch': 2.48}
+
 50%|████▉     | 372/750 [19:08<20:22,  3.23s/it]
 50%|████▉     | 373/750 [19:11<19:38,  3.13s/it]
                                                 
{'loss': 0.1933, 'grad_norm': 0.20800676941871643, 'learning_rate': 0.00012415937176812403, 'epoch': 2.49}
+
 50%|████▉     | 373/750 [19:11<19:38,  3.13s/it]
 50%|████▉     | 374/750 [19:14<19:18,  3.08s/it]
                                                 
{'loss': 0.2375, 'grad_norm': 0.21544986963272095, 'learning_rate': 0.0001239932721997534, 'epoch': 2.49}
+
 50%|████▉     | 374/750 [19:14<19:18,  3.08s/it]
 50%|█████     | 375/750 [19:17<19:19,  3.09s/it]
                                                 
{'loss': 0.176, 'grad_norm': 0.18405625224113464, 'learning_rate': 0.00012382783747337805, 'epoch': 2.5}
+
 50%|█████     | 375/750 [19:17<19:19,  3.09s/it]
 50%|█████     | 376/750 [19:20<19:50,  3.18s/it]
                                                 
{'loss': 0.2558, 'grad_norm': 0.22041290998458862, 'learning_rate': 0.0001236630631655603, 'epoch': 2.51}
+
 50%|█████     | 376/750 [19:20<19:50,  3.18s/it]
 50%|█████     | 377/750 [19:23<18:25,  2.96s/it]
                                                 
{'loss': 0.1964, 'grad_norm': 0.19243617355823517, 'learning_rate': 0.0001234989448939562, 'epoch': 2.51}
+
 50%|█████     | 377/750 [19:23<18:25,  2.96s/it]
 50%|█████     | 378/750 [19:25<16:25,  2.65s/it]
                                                 
{'loss': 0.2224, 'grad_norm': 0.20343562960624695, 'learning_rate': 0.00012333547831682581, 'epoch': 2.52}
+
 50%|█████     | 378/750 [19:25<16:25,  2.65s/it]
 51%|█████     | 379/750 [19:27<16:31,  2.67s/it]
                                                 
{'loss': 0.2142, 'grad_norm': 0.20934002101421356, 'learning_rate': 0.00012317265913255117, 'epoch': 2.53}
+
 51%|█████     | 379/750 [19:27<16:31,  2.67s/it]
 51%|█████     | 380/750 [19:30<16:35,  2.69s/it]
                                                 
{'loss': 0.1963, 'grad_norm': 0.19086094200611115, 'learning_rate': 0.00012301048307916047, 'epoch': 2.53}
+
 51%|█████     | 380/750 [19:30<16:35,  2.69s/it]
 51%|█████     | 381/750 [19:33<16:14,  2.64s/it]
                                                 
{'loss': 0.2011, 'grad_norm': 0.19029474258422852, 'learning_rate': 0.00012284894593385964, 'epoch': 2.54}
+
 51%|█████     | 381/750 [19:33<16:14,  2.64s/it]
 51%|█████     | 382/750 [19:35<16:13,  2.65s/it]
                                                 
{'loss': 0.2155, 'grad_norm': 0.21116392314434052, 'learning_rate': 0.00012268804351257058, 'epoch': 2.55}
+
 51%|█████     | 382/750 [19:35<16:13,  2.65s/it]
 51%|█████     | 383/750 [19:38<15:32,  2.54s/it]
                                                 
{'loss': 0.2165, 'grad_norm': 0.21682320535182953, 'learning_rate': 0.00012252777166947586, 'epoch': 2.55}
+
 51%|█████     | 383/750 [19:38<15:32,  2.54s/it]
 51%|█████     | 384/750 [19:41<16:12,  2.66s/it]
                                                 
{'loss': 0.1849, 'grad_norm': 0.1834099292755127, 'learning_rate': 0.0001223681262965701, 'epoch': 2.56}
+
 51%|█████     | 384/750 [19:41<16:12,  2.66s/it]
 51%|█████▏    | 385/750 [19:44<18:13,  3.00s/it]
                                                 
{'loss': 0.2009, 'grad_norm': 0.20737899839878082, 'learning_rate': 0.00012220910332321784, 'epoch': 2.57}
+
 51%|█████▏    | 385/750 [19:44<18:13,  3.00s/it]
 51%|█████▏    | 386/750 [19:47<17:36,  2.90s/it]
                                                 
{'loss': 0.2207, 'grad_norm': 0.2339901179075241, 'learning_rate': 0.00012205069871571739, 'epoch': 2.57}
+
 51%|█████▏    | 386/750 [19:47<17:36,  2.90s/it]
 52%|█████▏    | 387/750 [19:49<16:02,  2.65s/it]
                                                 
{'loss': 0.2197, 'grad_norm': 0.21467432379722595, 'learning_rate': 0.00012189290847687124, 'epoch': 2.58}
+
 52%|█████▏    | 387/750 [19:49<16:02,  2.65s/it]
 52%|█████▏    | 388/750 [19:52<16:05,  2.67s/it]
                                                 
{'loss': 0.1684, 'grad_norm': 0.1914362609386444, 'learning_rate': 0.0001217357286455623, 'epoch': 2.59}
+
 52%|█████▏    | 388/750 [19:52<16:05,  2.67s/it]
 52%|█████▏    | 389/750 [19:54<15:17,  2.54s/it]
                                                 
{'loss': 0.1756, 'grad_norm': 0.1855250746011734, 'learning_rate': 0.00012157915529633644, 'epoch': 2.59}
+
 52%|█████▏    | 389/750 [19:54<15:17,  2.54s/it]
 52%|█████▏    | 390/750 [19:56<14:43,  2.45s/it]
                                                 
{'loss': 0.2017, 'grad_norm': 0.21008794009685516, 'learning_rate': 0.00012142318453899053, 'epoch': 2.6}
+
 52%|█████▏    | 390/750 [19:56<14:43,  2.45s/it]
 52%|█████▏    | 391/750 [19:58<14:03,  2.35s/it]
                                                 
{'loss': 0.1889, 'grad_norm': 0.19353285431861877, 'learning_rate': 0.00012126781251816648, 'epoch': 2.61}
+
 52%|█████▏    | 391/750 [19:58<14:03,  2.35s/it]
 52%|█████▏    | 392/750 [20:00<13:10,  2.21s/it]
                                                 
{'loss': 0.2062, 'grad_norm': 0.21282115578651428, 'learning_rate': 0.00012111303541295122, 'epoch': 2.61}
+
 52%|█████▏    | 392/750 [20:00<13:10,  2.21s/it]
 52%|█████▏    | 393/750 [20:03<13:45,  2.31s/it]
                                                 
{'loss': 0.2325, 'grad_norm': 0.19696806371212006, 'learning_rate': 0.00012095884943648174, 'epoch': 2.62}
+
 52%|█████▏    | 393/750 [20:03<13:45,  2.31s/it]
 53%|█████▎    | 394/750 [20:06<14:53,  2.51s/it]
                                                 
{'loss': 0.2071, 'grad_norm': 0.19478800892829895, 'learning_rate': 0.0001208052508355561, 'epoch': 2.63}
+
 53%|█████▎    | 394/750 [20:06<14:53,  2.51s/it]
 53%|█████▎    | 395/750 [20:08<15:00,  2.54s/it]
                                                 
{'loss': 0.162, 'grad_norm': 0.17507150769233704, 'learning_rate': 0.0001206522358902497, 'epoch': 2.63}
+
 53%|█████▎    | 395/750 [20:08<15:00,  2.54s/it]
 53%|█████▎    | 396/750 [20:11<15:36,  2.65s/it]
                                                 
{'loss': 0.1616, 'grad_norm': 0.18554463982582092, 'learning_rate': 0.00012049980091353687, 'epoch': 2.64}
+
 53%|█████▎    | 396/750 [20:11<15:36,  2.65s/it]
 53%|█████▎    | 397/750 [20:13<14:37,  2.49s/it]
                                                 
{'loss': 0.2108, 'grad_norm': 0.23798343539237976, 'learning_rate': 0.00012034794225091773, 'epoch': 2.65}
+
 53%|█████▎    | 397/750 [20:13<14:37,  2.49s/it]
 53%|█████▎    | 398/750 [20:16<14:44,  2.51s/it]
                                                 
{'loss': 0.167, 'grad_norm': 0.17429287731647491, 'learning_rate': 0.00012019665628005017, 'epoch': 2.65}
+
 53%|█████▎    | 398/750 [20:16<14:44,  2.51s/it]
 53%|█████▎    | 399/750 [20:19<16:19,  2.79s/it]
                                                 
{'loss': 0.2066, 'grad_norm': 0.18508270382881165, 'learning_rate': 0.00012004593941038698, 'epoch': 2.66}
+
 53%|█████▎    | 399/750 [20:19<16:19,  2.79s/it]
 53%|█████▎    | 400/750 [20:25<21:16,  3.65s/it]
                                                 
{'loss': 0.1768, 'grad_norm': 0.1911124438047409, 'learning_rate': 0.00011989578808281799, 'epoch': 2.67}
+
 53%|█████▎    | 400/750 [20:25<21:16,  3.65s/it][INFO|trainer.py:3831] 2025-06-27 01:29:03,354 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-27 01:29:03,354 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-27 01:29:03,354 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:00<00:03,  2.35it/s]
+
 30%|███       | 3/10 [00:01<00:04,  1.72it/s]
+
 40%|████      | 4/10 [00:02<00:03,  1.61it/s]
+
 50%|█████     | 5/10 [00:03<00:04,  1.20it/s]
+
 60%|██████    | 6/10 [00:04<00:03,  1.21it/s]
+
 70%|███████   | 7/10 [00:05<00:02,  1.28it/s]
+
 80%|████████  | 8/10 [00:05<00:01,  1.40it/s]
+
 90%|█████████ | 9/10 [00:06<00:00,  1.30it/s]
+
100%|██████████| 10/10 [00:09<00:00,  1.44s/it]
                                                 
+
                                               
{'eval_loss': 0.2829940915107727, 'eval_accuracy': 0.4615254798146923, 'eval_runtime': 22.9081, 'eval_samples_per_second': 43.653, 'eval_steps_per_second': 0.437, 'epoch': 2.67}
+
 53%|█████▎    | 400/750 [20:48<21:16,  3.65s/it]
+
100%|██████████| 10/10 [00:13<00:00,  1.44s/it]
+
                                               /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[INFO|trainer.py:3515] 2025-06-27 01:29:40,336 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-400
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da6ac-518f920d3b37552b4adedab1;e0ba3fab-180f-4325-ac9d-5cb0720ea5aa)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-27 01:29:40,669 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-400/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-27 01:29:40,670 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-400/special_tokens_map.json
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[2025-06-27 01:29:41,797] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Checkpoint global_step400 is begin to save!
+[2025-06-27 01:29:41,823] [INFO] [logging.py:107:log_dist] [Rank 0] Saving model checkpoint: ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da6ae-66fb3ae43c7665de17253323;3150c20b-b193-4eb4-82fd-756e1c5b6da6)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da6ae-4da4f4653b524d4e776cdf87;237b5597-20c7-4204-80d3-bc83b682fca8)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da6ae-469fe4a66365f31b36fb17af;88d57355-bc86-4764-b24c-eb95edfa1a34)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da6ae-4e740fe632677b955b6e4d6a;4cf7364e-9d7f-4d4b-b35a-9bebf6c0bc37)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+
 53%|█████▎    | 401/750 [21:07<1:27:53, 15.11s/it]
                                                   
{'loss': 0.2548, 'grad_norm': 0.21075807511806488, 'learning_rate': 0.00011974619876931687, 'epoch': 2.67}
+
 53%|█████▎    | 401/750 [21:07<1:27:53, 15.11s/it]
 54%|█████▎    | 402/750 [21:10<1:05:58, 11.38s/it]
                                                   
{'loss': 0.2192, 'grad_norm': 0.235221728682518, 'learning_rate': 0.0001195971679725932, 'epoch': 2.68}
+
 54%|█████▎    | 402/750 [21:10<1:05:58, 11.38s/it]
 54%|█████▎    | 403/750 [21:13<51:16,  8.87s/it]  
                                                 
{'loss': 0.2, 'grad_norm': 0.20423269271850586, 'learning_rate': 0.00011944869222574892, 'epoch': 2.69}
+
 54%|█████▎    | 403/750 [21:13<51:16,  8.87s/it]
 54%|█████▍    | 404/750 [21:16<41:01,  7.11s/it]
                                                 
{'loss': 0.192, 'grad_norm': 0.19491294026374817, 'learning_rate': 0.00011930076809193951, 'epoch': 2.69}
+
 54%|█████▍    | 404/750 [21:16<41:01,  7.11s/it]
 54%|█████▍    | 405/750 [21:18<33:37,  5.85s/it]
                                                 
{'loss': 0.2153, 'grad_norm': 0.212855264544487, 'learning_rate': 0.0001191533921640401, 'epoch': 2.7}
+
 54%|█████▍    | 405/750 [21:18<33:37,  5.85s/it]
 54%|█████▍    | 406/750 [21:21<27:32,  4.80s/it]
                                                 
{'loss': 0.2143, 'grad_norm': 0.20352312922477722, 'learning_rate': 0.00011900656106431562, 'epoch': 2.71}
+
 54%|█████▍    | 406/750 [21:21<27:32,  4.80s/it]
 54%|█████▍    | 407/750 [21:23<22:58,  4.02s/it]
                                                 
{'loss': 0.1902, 'grad_norm': 0.19536025822162628, 'learning_rate': 0.00011886027144409578, 'epoch': 2.71}
+
 54%|█████▍    | 407/750 [21:23<22:58,  4.02s/it]
 54%|█████▍    | 408/750 [21:25<19:46,  3.47s/it]
                                                 
{'loss': 0.2288, 'grad_norm': 0.19938969612121582, 'learning_rate': 0.00011871451998345427, 'epoch': 2.72}
+
 54%|█████▍    | 408/750 [21:25<19:46,  3.47s/it]
 55%|█████▍    | 409/750 [21:27<17:18,  3.04s/it]
                                                 
{'loss': 0.2114, 'grad_norm': 0.2158292829990387, 'learning_rate': 0.00011856930339089229, 'epoch': 2.73}
+
 55%|█████▍    | 409/750 [21:27<17:18,  3.04s/it]
 55%|█████▍    | 410/750 [21:30<17:10,  3.03s/it]
                                                 
{'loss': 0.2573, 'grad_norm': 0.21224650740623474, 'learning_rate': 0.00011842461840302649, 'epoch': 2.73}
+
 55%|█████▍    | 410/750 [21:30<17:10,  3.03s/it]
 55%|█████▍    | 411/750 [21:33<16:28,  2.92s/it]
                                                 
{'loss': 0.1584, 'grad_norm': 0.17180979251861572, 'learning_rate': 0.00011828046178428064, 'epoch': 2.74}
+
 55%|█████▍    | 411/750 [21:33<16:28,  2.92s/it]
 55%|█████▍    | 412/750 [21:35<15:48,  2.81s/it]
                                                 
{'loss': 0.2023, 'grad_norm': 0.19157537817955017, 'learning_rate': 0.00011813683032658212, 'epoch': 2.75}
+
 55%|█████▍    | 412/750 [21:35<15:48,  2.81s/it]
 55%|█████▌    | 413/750 [21:38<15:46,  2.81s/it]
                                                 
{'loss': 0.197, 'grad_norm': 0.20072434842586517, 'learning_rate': 0.0001179937208490617, 'epoch': 2.75}
+
 55%|█████▌    | 413/750 [21:38<15:46,  2.81s/it]
 55%|█████▌    | 414/750 [21:42<17:33,  3.13s/it]
                                                 
{'loss': 0.2369, 'grad_norm': 0.22838325798511505, 'learning_rate': 0.00011785113019775794, 'epoch': 2.76}
+
 55%|█████▌    | 414/750 [21:42<17:33,  3.13s/it]
 55%|█████▌    | 415/750 [21:45<16:30,  2.96s/it]
                                                 
{'loss': 0.2357, 'grad_norm': 0.21541860699653625, 'learning_rate': 0.00011770905524532507, 'epoch': 2.77}
+
 55%|█████▌    | 415/750 [21:45<16:30,  2.96s/it]
 55%|█████▌    | 416/750 [21:47<15:27,  2.78s/it]
                                                 
{'loss': 0.2361, 'grad_norm': 0.222932830452919, 'learning_rate': 0.00011756749289074503, 'epoch': 2.77}
+
 55%|█████▌    | 416/750 [21:47<15:27,  2.78s/it]
 56%|█████▌    | 417/750 [21:50<16:27,  2.97s/it]
                                                 
{'loss': 0.1841, 'grad_norm': 0.1950065642595291, 'learning_rate': 0.00011742644005904313, 'epoch': 2.78}
+
 56%|█████▌    | 417/750 [21:50<16:27,  2.97s/it]
 56%|█████▌    | 418/750 [21:53<14:55,  2.70s/it]
                                                 
{'loss': 0.1925, 'grad_norm': 0.20776961743831635, 'learning_rate': 0.00011728589370100743, 'epoch': 2.79}
+
 56%|█████▌    | 418/750 [21:53<14:55,  2.70s/it]
 56%|█████▌    | 419/750 [21:55<14:13,  2.58s/it]
                                                 
{'loss': 0.2213, 'grad_norm': 0.23474028706550598, 'learning_rate': 0.00011714585079291212, 'epoch': 2.79}
+
 56%|█████▌    | 419/750 [21:55<14:13,  2.58s/it]
 56%|█████▌    | 420/750 [21:58<14:56,  2.72s/it]
                                                 
{'loss': 0.2379, 'grad_norm': 0.2087065577507019, 'learning_rate': 0.00011700630833624395, 'epoch': 2.8}
+
 56%|█████▌    | 420/750 [21:58<14:56,  2.72s/it]
 56%|█████▌    | 421/750 [22:00<14:40,  2.68s/it]
                                                 
{'loss': 0.2094, 'grad_norm': 0.232641339302063, 'learning_rate': 0.00011686726335743291, 'epoch': 2.81}
+
 56%|█████▌    | 421/750 [22:00<14:40,  2.68s/it]
 56%|█████▋    | 422/750 [22:04<15:44,  2.88s/it]
                                                 
{'loss': 0.2325, 'grad_norm': 0.1933550238609314, 'learning_rate': 0.0001167287129075859, 'epoch': 2.81}
+
 56%|█████▋    | 422/750 [22:04<15:44,  2.88s/it]
 56%|█████▋    | 423/750 [22:06<14:04,  2.58s/it]
                                                 
{'loss': 0.234, 'grad_norm': 0.2744564414024353, 'learning_rate': 0.00011659065406222409, 'epoch': 2.82}
+
 56%|█████▋    | 423/750 [22:06<14:04,  2.58s/it]
 57%|█████▋    | 424/750 [22:08<14:00,  2.58s/it]
                                                 
{'loss': 0.1786, 'grad_norm': 0.19435440003871918, 'learning_rate': 0.00011645308392102366, 'epoch': 2.83}
+
 57%|█████▋    | 424/750 [22:08<14:00,  2.58s/it]
 57%|█████▋    | 425/750 [22:11<14:09,  2.61s/it]
                                                 
{'loss': 0.1876, 'grad_norm': 0.1899508833885193, 'learning_rate': 0.00011631599960755992, 'epoch': 2.83}
+
 57%|█████▋    | 425/750 [22:11<14:09,  2.61s/it]
 57%|█████▋    | 426/750 [22:15<16:44,  3.10s/it]
                                                 
{'loss': 0.19, 'grad_norm': 0.18240424990653992, 'learning_rate': 0.00011617939826905469, 'epoch': 2.84}
+
 57%|█████▋    | 426/750 [22:15<16:44,  3.10s/it]
 57%|█████▋    | 427/750 [22:18<16:36,  3.09s/it]
                                                 
{'loss': 0.1879, 'grad_norm': 0.18841242790222168, 'learning_rate': 0.00011604327707612684, 'epoch': 2.85}
+
 57%|█████▋    | 427/750 [22:18<16:36,  3.09s/it]
 57%|█████▋    | 428/750 [22:21<16:26,  3.07s/it]
                                                 
{'loss': 0.1675, 'grad_norm': 0.17399099469184875, 'learning_rate': 0.00011590763322254638, 'epoch': 2.85}
+
 57%|█████▋    | 428/750 [22:21<16:26,  3.07s/it]
 57%|█████▋    | 429/750 [22:24<15:43,  2.94s/it]
                                                 
{'loss': 0.2031, 'grad_norm': 0.19952000677585602, 'learning_rate': 0.00011577246392499127, 'epoch': 2.86}
+
 57%|█████▋    | 429/750 [22:24<15:43,  2.94s/it]
 57%|█████▋    | 430/750 [22:27<15:16,  2.86s/it]
                                                 
{'loss': 0.1829, 'grad_norm': 0.21141472458839417, 'learning_rate': 0.00011563776642280763, 'epoch': 2.87}
+
 57%|█████▋    | 430/750 [22:27<15:16,  2.86s/it]
 57%|█████▋    | 431/750 [22:29<14:15,  2.68s/it]
                                                 
{'loss': 0.2062, 'grad_norm': 0.21844913065433502, 'learning_rate': 0.00011550353797777291, 'epoch': 2.87}
+
 57%|█████▋    | 431/750 [22:29<14:15,  2.68s/it]
 58%|█████▊    | 432/750 [22:32<15:24,  2.91s/it]
                                                 
{'loss': 0.1812, 'grad_norm': 0.20487137138843536, 'learning_rate': 0.00011536977587386216, 'epoch': 2.88}
+
 58%|█████▊    | 432/750 [22:32<15:24,  2.91s/it]
 58%|█████▊    | 433/750 [22:34<14:02,  2.66s/it]
                                                 
{'loss': 0.1901, 'grad_norm': 0.1970895379781723, 'learning_rate': 0.00011523647741701704, 'epoch': 2.89}
+
 58%|█████▊    | 433/750 [22:34<14:02,  2.66s/it]
 58%|█████▊    | 434/750 [22:38<14:45,  2.80s/it]
                                                 
{'loss': 0.2237, 'grad_norm': 0.21609656512737274, 'learning_rate': 0.00011510363993491814, 'epoch': 2.89}
+
 58%|█████▊    | 434/750 [22:38<14:45,  2.80s/it]
 58%|█████▊    | 435/750 [22:40<14:39,  2.79s/it]
                                                 
{'loss': 0.2154, 'grad_norm': 0.2267211228609085, 'learning_rate': 0.00011497126077675976, 'epoch': 2.9}
+
 58%|█████▊    | 435/750 [22:40<14:39,  2.79s/it]
 58%|█████▊    | 436/750 [22:46<18:56,  3.62s/it]
                                                 
{'loss': 0.2322, 'grad_norm': 0.20426948368549347, 'learning_rate': 0.00011483933731302795, 'epoch': 2.91}
+
 58%|█████▊    | 436/750 [22:46<18:56,  3.62s/it]
 58%|█████▊    | 437/750 [22:49<17:56,  3.44s/it]
                                                 
{'loss': 0.1929, 'grad_norm': 0.22252430021762848, 'learning_rate': 0.00011470786693528087, 'epoch': 2.91}
+
 58%|█████▊    | 437/750 [22:49<17:56,  3.44s/it]
 58%|█████▊    | 438/750 [22:54<21:10,  4.07s/it]
                                                 
{'loss': 0.2591, 'grad_norm': 0.23873890936374664, 'learning_rate': 0.00011457684705593247, 'epoch': 2.92}
+
 58%|█████▊    | 438/750 [22:54<21:10,  4.07s/it]
 59%|█████▊    | 439/750 [22:58<19:59,  3.86s/it]
                                                 
{'loss': 0.2261, 'grad_norm': 0.20575445890426636, 'learning_rate': 0.00011444627510803829, 'epoch': 2.93}
+
 59%|█████▊    | 439/750 [22:58<19:59,  3.86s/it]
 59%|█████▊    | 440/750 [23:00<17:32,  3.40s/it]
                                                 
{'loss': 0.1896, 'grad_norm': 0.19211649894714355, 'learning_rate': 0.00011431614854508447, 'epoch': 2.93}
+
 59%|█████▊    | 440/750 [23:00<17:32,  3.40s/it]
 59%|█████▉    | 441/750 [23:03<16:09,  3.14s/it]
                                                 
{'loss': 0.1566, 'grad_norm': 0.1832575649023056, 'learning_rate': 0.00011418646484077904, 'epoch': 2.94}
+
 59%|█████▉    | 441/750 [23:03<16:09,  3.14s/it]
 59%|█████▉    | 442/750 [23:06<15:58,  3.11s/it]
                                                 
{'loss': 0.2042, 'grad_norm': 0.19419528543949127, 'learning_rate': 0.00011405722148884592, 'epoch': 2.95}
+
 59%|█████▉    | 442/750 [23:06<15:58,  3.11s/it]
 59%|█████▉    | 443/750 [23:09<15:29,  3.03s/it]
                                                 
{'loss': 0.206, 'grad_norm': 0.20362479984760284, 'learning_rate': 0.00011392841600282156, 'epoch': 2.95}
+
 59%|█████▉    | 443/750 [23:09<15:29,  3.03s/it]
 59%|█████▉    | 444/750 [23:10<13:38,  2.67s/it]
                                                 
{'loss': 0.2725, 'grad_norm': 0.22774358093738556, 'learning_rate': 0.00011380004591585387, 'epoch': 2.96}
+
 59%|█████▉    | 444/750 [23:10<13:38,  2.67s/it]
 59%|█████▉    | 445/750 [23:14<14:41,  2.89s/it]
                                                 
{'loss': 0.2021, 'grad_norm': 0.20395444333553314, 'learning_rate': 0.00011367210878050402, 'epoch': 2.97}
+
 59%|█████▉    | 445/750 [23:14<14:41,  2.89s/it]
 59%|█████▉    | 446/750 [23:16<14:06,  2.78s/it]
                                                 
{'loss': 0.176, 'grad_norm': 0.19900578260421753, 'learning_rate': 0.00011354460216855028, 'epoch': 2.97}
+
 59%|█████▉    | 446/750 [23:16<14:06,  2.78s/it]
 60%|█████▉    | 447/750 [23:20<14:46,  2.93s/it]
                                                 
{'loss': 0.2041, 'grad_norm': 0.18975220620632172, 'learning_rate': 0.00011341752367079442, 'epoch': 2.98}
+
 60%|█████▉    | 447/750 [23:20<14:46,  2.93s/it]
 60%|█████▉    | 448/750 [23:22<14:28,  2.87s/it]
                                                 
{'loss': 0.2133, 'grad_norm': 0.20021529495716095, 'learning_rate': 0.00011329087089687071, 'epoch': 2.99}
+
 60%|█████▉    | 448/750 [23:22<14:28,  2.87s/it]
 60%|█████▉    | 449/750 [23:25<13:59,  2.79s/it]
                                                 
{'loss': 0.227, 'grad_norm': 0.23890961706638336, 'learning_rate': 0.00011316464147505693, 'epoch': 2.99}
+
 60%|█████▉    | 449/750 [23:25<13:59,  2.79s/it]
 60%|██████    | 450/750 [23:31<18:20,  3.67s/it]
                                                 
{'loss': 0.2165, 'grad_norm': 0.19924578070640564, 'learning_rate': 0.0001130388330520878, 'epoch': 3.0}
+
 60%|██████    | 450/750 [23:31<18:20,  3.67s/it]
 60%|██████    | 451/750 [23:43<30:40,  6.15s/it]
                                                 
{'loss': 0.1828, 'grad_norm': 0.21424797177314758, 'learning_rate': 0.00011291344329297086, 'epoch': 3.01}
+
 60%|██████    | 451/750 [23:43<30:40,  6.15s/it]
 60%|██████    | 452/750 [23:46<25:49,  5.20s/it]
                                                 
{'loss': 0.1544, 'grad_norm': 0.19907167553901672, 'learning_rate': 0.0001127884698808043, 'epoch': 3.01}
+
 60%|██████    | 452/750 [23:46<25:49,  5.20s/it]
 60%|██████    | 453/750 [23:48<22:00,  4.45s/it]
                                                 
{'loss': 0.152, 'grad_norm': 0.2020040899515152, 'learning_rate': 0.00011266391051659725, 'epoch': 3.02}
+
 60%|██████    | 453/750 [23:48<22:00,  4.45s/it]
 61%|██████    | 454/750 [23:51<19:25,  3.94s/it]
                                                 
{'loss': 0.1305, 'grad_norm': 0.18440668284893036, 'learning_rate': 0.00011253976291909212, 'epoch': 3.03}
+
 61%|██████    | 454/750 [23:51<19:25,  3.94s/it]
 61%|██████    | 455/750 [23:54<17:32,  3.57s/it]
                                                 
{'loss': 0.1346, 'grad_norm': 0.19786520302295685, 'learning_rate': 0.00011241602482458911, 'epoch': 3.03}
+
 61%|██████    | 455/750 [23:54<17:32,  3.57s/it]
 61%|██████    | 456/750 [23:59<20:21,  4.15s/it]
                                                 
{'loss': 0.1359, 'grad_norm': 0.20893609523773193, 'learning_rate': 0.00011229269398677288, 'epoch': 3.04}
+
 61%|██████    | 456/750 [23:59<20:21,  4.15s/it]
 61%|██████    | 457/750 [24:02<17:36,  3.60s/it]
                                                 
{'loss': 0.1349, 'grad_norm': 0.25663018226623535, 'learning_rate': 0.00011216976817654119, 'epoch': 3.05}
+
 61%|██████    | 457/750 [24:02<17:36,  3.60s/it]
 61%|██████    | 458/750 [24:04<16:06,  3.31s/it]
                                                 
{'loss': 0.1546, 'grad_norm': 0.30330052971839905, 'learning_rate': 0.00011204724518183582, 'epoch': 3.05}
+
 61%|██████    | 458/750 [24:04<16:06,  3.31s/it]
 61%|██████    | 459/750 [24:08<16:09,  3.33s/it]
                                                 
{'loss': 0.1595, 'grad_norm': 0.26031461358070374, 'learning_rate': 0.00011192512280747522, 'epoch': 3.06}
+
 61%|██████    | 459/750 [24:08<16:09,  3.33s/it]
 61%|██████▏   | 460/750 [24:10<14:54,  3.08s/it]
                                                 
{'loss': 0.1861, 'grad_norm': 0.2949683666229248, 'learning_rate': 0.00011180339887498949, 'epoch': 3.07}
+
 61%|██████▏   | 460/750 [24:10<14:54,  3.08s/it]
 61%|██████▏   | 461/750 [24:13<14:17,  2.97s/it]
                                                 
{'loss': 0.1754, 'grad_norm': 0.23577575385570526, 'learning_rate': 0.000111682071222457, 'epoch': 3.07}
+
 61%|██████▏   | 461/750 [24:13<14:17,  2.97s/it]
 62%|██████▏   | 462/750 [24:15<13:48,  2.88s/it]
                                                 
{'loss': 0.1863, 'grad_norm': 0.26501795649528503, 'learning_rate': 0.00011156113770434329, 'epoch': 3.08}
+
 62%|██████▏   | 462/750 [24:15<13:48,  2.88s/it]
 62%|██████▏   | 463/750 [24:20<15:43,  3.29s/it]
                                                 
{'loss': 0.1324, 'grad_norm': 0.20267081260681152, 'learning_rate': 0.00011144059619134151, 'epoch': 3.09}
+
 62%|██████▏   | 463/750 [24:20<15:43,  3.29s/it]
 62%|██████▏   | 464/750 [24:22<14:51,  3.12s/it]
                                                 
{'loss': 0.1483, 'grad_norm': 0.22169460356235504, 'learning_rate': 0.00011132044457021517, 'epoch': 3.09}
+
 62%|██████▏   | 464/750 [24:22<14:51,  3.12s/it]
 62%|██████▏   | 465/750 [24:25<13:54,  2.93s/it]
                                                 
{'loss': 0.1338, 'grad_norm': 0.22177430987358093, 'learning_rate': 0.00011120068074364232, 'epoch': 3.1}
+
 62%|██████▏   | 465/750 [24:25<13:54,  2.93s/it]
 62%|██████▏   | 466/750 [24:27<12:59,  2.75s/it]
                                                 
{'loss': 0.1597, 'grad_norm': 0.2297315001487732, 'learning_rate': 0.000111081302630062, 'epoch': 3.11}
+
 62%|██████▏   | 466/750 [24:27<12:59,  2.75s/it]
 62%|██████▏   | 467/750 [24:30<12:59,  2.76s/it]
                                                 
{'loss': 0.1505, 'grad_norm': 0.21048645675182343, 'learning_rate': 0.0001109623081635221, 'epoch': 3.11}
+
 62%|██████▏   | 467/750 [24:30<12:59,  2.76s/it]
 62%|██████▏   | 468/750 [24:36<17:12,  3.66s/it]
                                                 
{'loss': 0.2013, 'grad_norm': 0.2419176697731018, 'learning_rate': 0.00011084369529352937, 'epoch': 3.12}
+
 62%|██████▏   | 468/750 [24:36<17:12,  3.66s/it]
 63%|██████▎   | 469/750 [24:38<15:42,  3.36s/it]
                                                 
{'loss': 0.1456, 'grad_norm': 0.2442295104265213, 'learning_rate': 0.00011072546198490086, 'epoch': 3.13}
+
 63%|██████▎   | 469/750 [24:38<15:42,  3.36s/it]
 63%|██████▎   | 470/750 [24:41<14:51,  3.18s/it]
                                                 
{'loss': 0.148, 'grad_norm': 0.2591129243373871, 'learning_rate': 0.00011060760621761728, 'epoch': 3.13}
+
 63%|██████▎   | 470/750 [24:41<14:51,  3.18s/it]
 63%|██████▎   | 471/750 [24:43<13:21,  2.87s/it]
                                                 
{'loss': 0.1737, 'grad_norm': 0.25614669919013977, 'learning_rate': 0.00011049012598667817, 'epoch': 3.14}
+
 63%|██████▎   | 471/750 [24:43<13:21,  2.87s/it]
 63%|██████▎   | 472/750 [24:46<13:11,  2.85s/it]
                                                 
{'loss': 0.155, 'grad_norm': 0.2713840901851654, 'learning_rate': 0.00011037301930195852, 'epoch': 3.15}
+
 63%|██████▎   | 472/750 [24:46<13:11,  2.85s/it]
 63%|██████▎   | 473/750 [24:48<12:26,  2.70s/it]
                                                 
{'loss': 0.1452, 'grad_norm': 0.32019785046577454, 'learning_rate': 0.00011025628418806717, 'epoch': 3.15}
+
 63%|██████▎   | 473/750 [24:48<12:26,  2.70s/it]
 63%|██████▎   | 474/750 [24:52<13:16,  2.89s/it]
                                                 
{'loss': 0.1582, 'grad_norm': 0.24277234077453613, 'learning_rate': 0.00011013991868420692, 'epoch': 3.16}
+
 63%|██████▎   | 474/750 [24:52<13:16,  2.89s/it]
 63%|██████▎   | 475/750 [24:55<13:09,  2.87s/it]
                                                 
{'loss': 0.1269, 'grad_norm': 0.2201850265264511, 'learning_rate': 0.00011002392084403617, 'epoch': 3.17}
+
 63%|██████▎   | 475/750 [24:55<13:09,  2.87s/it]
 63%|██████▎   | 476/750 [24:57<12:28,  2.73s/it]
                                                 
{'loss': 0.1628, 'grad_norm': 0.2675172686576843, 'learning_rate': 0.00010990828873553218, 'epoch': 3.17}
+
 63%|██████▎   | 476/750 [24:57<12:28,  2.73s/it]
 64%|██████▎   | 477/750 [25:00<12:59,  2.86s/it]
                                                 
{'loss': 0.1568, 'grad_norm': 0.2519579827785492, 'learning_rate': 0.00010979302044085591, 'epoch': 3.18}
+
 64%|██████▎   | 477/750 [25:00<12:59,  2.86s/it]
 64%|██████▎   | 478/750 [25:03<12:34,  2.77s/it]
                                                 
{'loss': 0.1659, 'grad_norm': 0.24592290818691254, 'learning_rate': 0.00010967811405621847, 'epoch': 3.19}
+
 64%|██████▎   | 478/750 [25:03<12:34,  2.77s/it]
 64%|██████▍   | 479/750 [25:06<13:04,  2.90s/it]
                                                 
{'loss': 0.1413, 'grad_norm': 0.20691260695457458, 'learning_rate': 0.000109563567691749, 'epoch': 3.19}
+
 64%|██████▍   | 479/750 [25:06<13:04,  2.90s/it]
 64%|██████▍   | 480/750 [25:08<11:47,  2.62s/it]
                                                 
{'loss': 0.1677, 'grad_norm': 0.255313515663147, 'learning_rate': 0.00010944937947136412, 'epoch': 3.2}
+
 64%|██████▍   | 480/750 [25:08<11:47,  2.62s/it]
 64%|██████▍   | 481/750 [25:10<11:28,  2.56s/it]
                                                 
{'loss': 0.1529, 'grad_norm': 0.2259657382965088, 'learning_rate': 0.00010933554753263895, 'epoch': 3.21}
+
 64%|██████▍   | 481/750 [25:10<11:28,  2.56s/it]
 64%|██████▍   | 482/750 [25:13<11:23,  2.55s/it]
                                                 
{'loss': 0.133, 'grad_norm': 0.2348180115222931, 'learning_rate': 0.00010922207002667943, 'epoch': 3.21}
+
 64%|██████▍   | 482/750 [25:13<11:23,  2.55s/it]
 64%|██████▍   | 483/750 [25:15<11:02,  2.48s/it]
                                                 
{'loss': 0.1491, 'grad_norm': 0.24075885117053986, 'learning_rate': 0.0001091089451179962, 'epoch': 3.22}
+
 64%|██████▍   | 483/750 [25:15<11:02,  2.48s/it]
 65%|██████▍   | 484/750 [25:19<13:25,  3.03s/it]
                                                 
{'loss': 0.1785, 'grad_norm': 0.25071612000465393, 'learning_rate': 0.00010899617098437999, 'epoch': 3.23}
+
 65%|██████▍   | 484/750 [25:19<13:25,  3.03s/it]
 65%|██████▍   | 485/750 [25:22<12:47,  2.90s/it]
                                                 
{'loss': 0.1646, 'grad_norm': 0.23533204197883606, 'learning_rate': 0.00010888374581677828, 'epoch': 3.23}
+
 65%|██████▍   | 485/750 [25:22<12:47,  2.90s/it]
 65%|██████▍   | 486/750 [25:24<12:07,  2.75s/it]
                                                 
{'loss': 0.1585, 'grad_norm': 0.2626405358314514, 'learning_rate': 0.00010877166781917343, 'epoch': 3.24}
+
 65%|██████▍   | 486/750 [25:24<12:07,  2.75s/it]
 65%|██████▍   | 487/750 [25:29<14:53,  3.40s/it]
                                                 
{'loss': 0.1603, 'grad_norm': 0.2568136155605316, 'learning_rate': 0.00010865993520846211, 'epoch': 3.25}
+
 65%|██████▍   | 487/750 [25:29<14:53,  3.40s/it]
 65%|██████▌   | 488/750 [25:32<13:53,  3.18s/it]
                                                 
{'loss': 0.1514, 'grad_norm': 0.26485803723335266, 'learning_rate': 0.0001085485462143363, 'epoch': 3.25}
+
 65%|██████▌   | 488/750 [25:32<13:53,  3.18s/it]
 65%|██████▌   | 489/750 [25:35<13:26,  3.09s/it]
                                                 
{'loss': 0.1301, 'grad_norm': 0.24729150533676147, 'learning_rate': 0.00010843749907916526, 'epoch': 3.26}
+
 65%|██████▌   | 489/750 [25:35<13:26,  3.09s/it]
 65%|██████▌   | 490/750 [25:38<12:49,  2.96s/it]
                                                 
{'loss': 0.1657, 'grad_norm': 0.2655632495880127, 'learning_rate': 0.0001083267920578793, 'epoch': 3.27}
+
 65%|██████▌   | 490/750 [25:38<12:49,  2.96s/it]
 65%|██████▌   | 491/750 [25:40<12:12,  2.83s/it]
                                                 
{'loss': 0.1401, 'grad_norm': 0.229946106672287, 'learning_rate': 0.00010821642341785439, 'epoch': 3.27}
+
 65%|██████▌   | 491/750 [25:40<12:12,  2.83s/it]
 66%|██████▌   | 492/750 [25:43<11:42,  2.72s/it]
                                                 
{'loss': 0.165, 'grad_norm': 0.2663438618183136, 'learning_rate': 0.00010810639143879838, 'epoch': 3.28}
+
 66%|██████▌   | 492/750 [25:43<11:42,  2.72s/it]
 66%|██████▌   | 493/750 [25:46<11:57,  2.79s/it]
                                                 
{'loss': 0.1507, 'grad_norm': 0.24755999445915222, 'learning_rate': 0.00010799669441263838, 'epoch': 3.29}
+
 66%|██████▌   | 493/750 [25:46<11:57,  2.79s/it]
 66%|██████▌   | 494/750 [25:47<10:50,  2.54s/it]
                                                 
{'loss': 0.1865, 'grad_norm': 0.28706657886505127, 'learning_rate': 0.0001078873306434093, 'epoch': 3.29}
+
 66%|██████▌   | 494/750 [25:47<10:50,  2.54s/it]
 66%|██████▌   | 495/750 [25:50<10:26,  2.46s/it]
                                                 
{'loss': 0.1726, 'grad_norm': 0.28298503160476685, 'learning_rate': 0.00010777829844714388, 'epoch': 3.3}
+
 66%|██████▌   | 495/750 [25:50<10:26,  2.46s/it]
 66%|██████▌   | 496/750 [25:52<10:22,  2.45s/it]
                                                 
{'loss': 0.1379, 'grad_norm': 0.226507306098938, 'learning_rate': 0.0001076695961517636, 'epoch': 3.31}
+
 66%|██████▌   | 496/750 [25:52<10:22,  2.45s/it]
 66%|██████▋   | 497/750 [25:55<10:28,  2.48s/it]
                                                 
{'loss': 0.1419, 'grad_norm': 0.28493958711624146, 'learning_rate': 0.0001075612220969712, 'epoch': 3.31}
+
 66%|██████▋   | 497/750 [25:55<10:28,  2.48s/it]
 66%|██████▋   | 498/750 [25:57<10:09,  2.42s/it]
                                                 
{'loss': 0.1385, 'grad_norm': 0.2627260982990265, 'learning_rate': 0.00010745317463414389, 'epoch': 3.32}
+
 66%|██████▋   | 498/750 [25:57<10:09,  2.42s/it]
 67%|██████▋   | 499/750 [26:00<10:19,  2.47s/it]
                                                 
{'loss': 0.1373, 'grad_norm': 0.22178365290164948, 'learning_rate': 0.00010734545212622828, 'epoch': 3.33}
+
 67%|██████▋   | 499/750 [26:00<10:19,  2.47s/it]
 67%|██████▋   | 500/750 [26:02<10:45,  2.58s/it]
                                                 
{'loss': 0.1739, 'grad_norm': 0.25754880905151367, 'learning_rate': 0.00010723805294763608, 'epoch': 3.33}
+
 67%|██████▋   | 500/750 [26:02<10:45,  2.58s/it]
 67%|██████▋   | 501/750 [26:05<10:39,  2.57s/it]
                                                 
{'loss': 0.1903, 'grad_norm': 0.2821175754070282, 'learning_rate': 0.00010713097548414107, 'epoch': 3.34}
+
 67%|██████▋   | 501/750 [26:05<10:39,  2.57s/it]
 67%|██████▋   | 502/750 [26:08<10:39,  2.58s/it]
                                                 
{'loss': 0.1709, 'grad_norm': 0.28239837288856506, 'learning_rate': 0.00010702421813277718, 'epoch': 3.35}
+
 67%|██████▋   | 502/750 [26:08<10:39,  2.58s/it]
 67%|██████▋   | 503/750 [26:10<11:00,  2.68s/it]
                                                 
{'loss': 0.1896, 'grad_norm': 0.2821494936943054, 'learning_rate': 0.0001069177793017378, 'epoch': 3.35}
+
 67%|██████▋   | 503/750 [26:10<11:00,  2.68s/it]
 67%|██████▋   | 504/750 [26:13<10:21,  2.53s/it]
                                                 
{'loss': 0.1857, 'grad_norm': 0.28943175077438354, 'learning_rate': 0.00010681165741027595, 'epoch': 3.36}
+
 67%|██████▋   | 504/750 [26:13<10:21,  2.53s/it]
 67%|██████▋   | 505/750 [26:15<10:18,  2.52s/it]
                                                 
{'loss': 0.1732, 'grad_norm': 0.2506386935710907, 'learning_rate': 0.00010670585088860586, 'epoch': 3.37}
+
 67%|██████▋   | 505/750 [26:15<10:18,  2.52s/it]
 67%|██████▋   | 506/750 [26:18<10:17,  2.53s/it]
                                                 
{'loss': 0.1752, 'grad_norm': 0.2932261824607849, 'learning_rate': 0.00010660035817780521, 'epoch': 3.37}
+
 67%|██████▋   | 506/750 [26:18<10:17,  2.53s/it]
 68%|██████▊   | 507/750 [26:22<12:03,  2.98s/it]
                                                 
{'loss': 0.1389, 'grad_norm': 0.2402878850698471, 'learning_rate': 0.00010649517772971891, 'epoch': 3.38}
+
 68%|██████▊   | 507/750 [26:22<12:03,  2.98s/it]
 68%|██████▊   | 508/750 [26:26<13:04,  3.24s/it]
                                                 
{'loss': 0.1367, 'grad_norm': 0.24524889886379242, 'learning_rate': 0.00010639030800686347, 'epoch': 3.39}
+
 68%|██████▊   | 508/750 [26:26<13:04,  3.24s/it]
 68%|██████▊   | 509/750 [26:31<15:54,  3.96s/it]
                                                 
{'loss': 0.1439, 'grad_norm': 0.23677967488765717, 'learning_rate': 0.0001062857474823327, 'epoch': 3.39}
+
 68%|██████▊   | 509/750 [26:31<15:54,  3.96s/it]
 68%|██████▊   | 510/750 [26:33<13:41,  3.42s/it]
                                                 
{'loss': 0.152, 'grad_norm': 0.2505894601345062, 'learning_rate': 0.00010618149463970438, 'epoch': 3.4}
+
 68%|██████▊   | 510/750 [26:33<13:41,  3.42s/it]
 68%|██████▊   | 511/750 [26:36<12:48,  3.22s/it]
                                                 
{'loss': 0.1317, 'grad_norm': 0.2240130752325058, 'learning_rate': 0.00010607754797294777, 'epoch': 3.41}
+
 68%|██████▊   | 511/750 [26:36<12:48,  3.22s/it]
 68%|██████▊   | 512/750 [26:39<11:45,  2.97s/it]
                                                 
{'loss': 0.1743, 'grad_norm': 0.2639698386192322, 'learning_rate': 0.00010597390598633231, 'epoch': 3.41}
+
 68%|██████▊   | 512/750 [26:39<11:45,  2.97s/it]
 68%|██████▊   | 513/750 [26:41<10:51,  2.75s/it]
                                                 
{'loss': 0.1601, 'grad_norm': 0.2874159812927246, 'learning_rate': 0.00010587056719433728, 'epoch': 3.42}
+
 68%|██████▊   | 513/750 [26:41<10:51,  2.75s/it]
 69%|██████▊   | 514/750 [26:44<11:41,  2.97s/it]
                                                 
{'loss': 0.1915, 'grad_norm': 0.26162558794021606, 'learning_rate': 0.00010576753012156217, 'epoch': 3.43}
+
 69%|██████▊   | 514/750 [26:44<11:41,  2.97s/it]
 69%|██████▊   | 515/750 [26:47<11:25,  2.92s/it]
                                                 
{'loss': 0.1758, 'grad_norm': 0.3080141544342041, 'learning_rate': 0.0001056647933026385, 'epoch': 3.43}
+
 69%|██████▊   | 515/750 [26:47<11:25,  2.92s/it]
 69%|██████▉   | 516/750 [26:50<11:30,  2.95s/it]
                                                 
{'loss': 0.1733, 'grad_norm': 0.28200358152389526, 'learning_rate': 0.00010556235528214203, 'epoch': 3.44}
+
 69%|██████▉   | 516/750 [26:50<11:30,  2.95s/it]
 69%|██████▉   | 517/750 [26:56<14:30,  3.74s/it]
                                                 
{'loss': 0.1736, 'grad_norm': 0.24935802817344666, 'learning_rate': 0.00010546021461450624, 'epoch': 3.45}
+
 69%|██████▉   | 517/750 [26:56<14:30,  3.74s/it]
 69%|██████▉   | 518/750 [26:58<13:14,  3.43s/it]
                                                 
{'loss': 0.1686, 'grad_norm': 0.2666882872581482, 'learning_rate': 0.00010535836986393678, 'epoch': 3.45}
+
 69%|██████▉   | 518/750 [26:58<13:14,  3.43s/it]
 69%|██████▉   | 519/750 [27:02<13:52,  3.60s/it]
                                                 
{'loss': 0.1787, 'grad_norm': 0.2801056206226349, 'learning_rate': 0.00010525681960432646, 'epoch': 3.46}
+
 69%|██████▉   | 519/750 [27:02<13:52,  3.60s/it]
 69%|██████▉   | 520/750 [27:06<13:32,  3.53s/it]
                                                 
{'loss': 0.1601, 'grad_norm': 0.2656433880329132, 'learning_rate': 0.00010515556241917166, 'epoch': 3.47}
+
 69%|██████▉   | 520/750 [27:06<13:32,  3.53s/it]
 69%|██████▉   | 521/750 [27:08<12:30,  3.28s/it]
                                                 
{'loss': 0.1979, 'grad_norm': 0.27216312289237976, 'learning_rate': 0.00010505459690148908, 'epoch': 3.47}
+
 69%|██████▉   | 521/750 [27:08<12:30,  3.28s/it]
 70%|██████▉   | 522/750 [27:12<12:34,  3.31s/it]
                                                 
{'loss': 0.1545, 'grad_norm': 0.2471679300069809, 'learning_rate': 0.00010495392165373376, 'epoch': 3.48}
+
 70%|██████▉   | 522/750 [27:12<12:34,  3.31s/it]
 70%|██████▉   | 523/750 [27:14<11:37,  3.07s/it]
                                                 
{'loss': 0.1644, 'grad_norm': 0.2841789126396179, 'learning_rate': 0.00010485353528771787, 'epoch': 3.49}
+
 70%|██████▉   | 523/750 [27:14<11:37,  3.07s/it]
 70%|██████▉   | 524/750 [27:16<10:19,  2.74s/it]
                                                 
{'loss': 0.1379, 'grad_norm': 0.2629833221435547, 'learning_rate': 0.00010475343642453021, 'epoch': 3.49}
+
 70%|██████▉   | 524/750 [27:16<10:19,  2.74s/it]
 70%|███████   | 525/750 [27:19<10:16,  2.74s/it]
                                                 
{'loss': 0.1387, 'grad_norm': 0.23826737701892853, 'learning_rate': 0.00010465362369445672, 'epoch': 3.5}
+
 70%|███████   | 525/750 [27:19<10:16,  2.74s/it]
 70%|███████   | 526/750 [27:22<10:09,  2.72s/it]
                                                 
{'loss': 0.1615, 'grad_norm': 0.25253307819366455, 'learning_rate': 0.00010455409573690175, 'epoch': 3.51}
+
 70%|███████   | 526/750 [27:22<10:09,  2.72s/it]
 70%|███████   | 527/750 [27:24<10:03,  2.71s/it]
                                                 
{'loss': 0.1592, 'grad_norm': 0.2847226560115814, 'learning_rate': 0.00010445485120031016, 'epoch': 3.51}
+
 70%|███████   | 527/750 [27:24<10:03,  2.71s/it]
 70%|███████   | 528/750 [27:27<10:26,  2.82s/it]
                                                 
{'loss': 0.1537, 'grad_norm': 0.2696542739868164, 'learning_rate': 0.00010435588874209024, 'epoch': 3.52}
+
 70%|███████   | 528/750 [27:27<10:26,  2.82s/it]
 71%|███████   | 529/750 [27:31<10:55,  2.97s/it]
                                                 
{'loss': 0.1358, 'grad_norm': 0.25416213274002075, 'learning_rate': 0.00010425720702853738, 'epoch': 3.53}
+
 71%|███████   | 529/750 [27:31<10:55,  2.97s/it]
 71%|███████   | 530/750 [27:34<10:37,  2.90s/it]
                                                 
{'loss': 0.1749, 'grad_norm': 0.26647642254829407, 'learning_rate': 0.00010415880473475868, 'epoch': 3.53}
+
 71%|███████   | 530/750 [27:34<10:37,  2.90s/it]
 71%|███████   | 531/750 [27:36<10:37,  2.91s/it]
                                                 
{'loss': 0.148, 'grad_norm': 0.2207992672920227, 'learning_rate': 0.0001040606805445981, 'epoch': 3.54}
+
 71%|███████   | 531/750 [27:36<10:37,  2.91s/it]
 71%|███████   | 532/750 [27:40<10:45,  2.96s/it]
                                                 
{'loss': 0.1668, 'grad_norm': 0.2413429170846939, 'learning_rate': 0.00010396283315056266, 'epoch': 3.55}
+
 71%|███████   | 532/750 [27:40<10:45,  2.96s/it]
 71%|███████   | 533/750 [27:43<11:33,  3.20s/it]
                                                 
{'loss': 0.1449, 'grad_norm': 0.22839143872261047, 'learning_rate': 0.00010386526125374917, 'epoch': 3.55}
+
 71%|███████   | 533/750 [27:43<11:33,  3.20s/it]
 71%|███████   | 534/750 [27:46<10:49,  3.01s/it]
                                                 
{'loss': 0.1707, 'grad_norm': 0.2791605293750763, 'learning_rate': 0.00010376796356377186, 'epoch': 3.56}
+
 71%|███████   | 534/750 [27:46<10:49,  3.01s/it]
 71%|███████▏  | 535/750 [27:49<10:56,  3.05s/it]
                                                 
{'loss': 0.1617, 'grad_norm': 0.25681227445602417, 'learning_rate': 0.00010367093879869068, 'epoch': 3.57}
+
 71%|███████▏  | 535/750 [27:49<10:56,  3.05s/it]
 71%|███████▏  | 536/750 [27:52<10:44,  3.01s/it]
                                                 
{'loss': 0.1801, 'grad_norm': 0.2867075204849243, 'learning_rate': 0.00010357418568494037, 'epoch': 3.57}
+
 71%|███████▏  | 536/750 [27:52<10:44,  3.01s/it]
 72%|███████▏  | 537/750 [27:55<10:32,  2.97s/it]
                                                 
{'loss': 0.1533, 'grad_norm': 0.2677098214626312, 'learning_rate': 0.00010347770295726023, 'epoch': 3.58}
+
 72%|███████▏  | 537/750 [27:55<10:32,  2.97s/it]
 72%|███████▏  | 538/750 [27:58<10:49,  3.06s/it]
                                                 
{'loss': 0.1454, 'grad_norm': 0.2393018752336502, 'learning_rate': 0.00010338148935862473, 'epoch': 3.59}
+
 72%|███████▏  | 538/750 [27:58<10:49,  3.06s/it]
 72%|███████▏  | 539/750 [28:00<09:54,  2.82s/it]
                                                 
{'loss': 0.171, 'grad_norm': 0.26799800992012024, 'learning_rate': 0.00010328554364017445, 'epoch': 3.59}
+
 72%|███████▏  | 539/750 [28:00<09:54,  2.82s/it]
 72%|███████▏  | 540/750 [28:03<10:03,  2.88s/it]
                                                 
{'loss': 0.1588, 'grad_norm': 0.2654531002044678, 'learning_rate': 0.00010318986456114837, 'epoch': 3.6}
+
 72%|███████▏  | 540/750 [28:03<10:03,  2.88s/it]
 72%|███████▏  | 541/750 [28:05<09:14,  2.65s/it]
                                                 
{'loss': 0.1634, 'grad_norm': 0.2694476842880249, 'learning_rate': 0.00010309445088881625, 'epoch': 3.61}
+
 72%|███████▏  | 541/750 [28:05<09:14,  2.65s/it]
 72%|███████▏  | 542/750 [28:07<08:33,  2.47s/it]
                                                 
{'loss': 0.1772, 'grad_norm': 0.29812100529670715, 'learning_rate': 0.00010299930139841185, 'epoch': 3.61}
+
 72%|███████▏  | 542/750 [28:07<08:33,  2.47s/it]
 72%|███████▏  | 543/750 [28:11<09:27,  2.74s/it]
                                                 
{'loss': 0.1485, 'grad_norm': 0.24032343924045563, 'learning_rate': 0.00010290441487306716, 'epoch': 3.62}
+
 72%|███████▏  | 543/750 [28:11<09:27,  2.74s/it]
 73%|███████▎  | 544/750 [28:13<08:51,  2.58s/it]
                                                 
{'loss': 0.1385, 'grad_norm': 0.2727023959159851, 'learning_rate': 0.00010280979010374679, 'epoch': 3.63}
+
 73%|███████▎  | 544/750 [28:13<08:51,  2.58s/it]
 73%|███████▎  | 545/750 [28:15<08:36,  2.52s/it]
                                                 
{'loss': 0.1553, 'grad_norm': 0.288371205329895, 'learning_rate': 0.00010271542588918341, 'epoch': 3.63}
+
 73%|███████▎  | 545/750 [28:15<08:36,  2.52s/it]
 73%|███████▎  | 546/750 [28:19<09:38,  2.84s/it]
                                                 
{'loss': 0.1403, 'grad_norm': 0.2412414848804474, 'learning_rate': 0.00010262132103581367, 'epoch': 3.64}
+
 73%|███████▎  | 546/750 [28:19<09:38,  2.84s/it]
 73%|███████▎  | 547/750 [28:22<09:51,  2.91s/it]
                                                 
{'loss': 0.1534, 'grad_norm': 0.24089080095291138, 'learning_rate': 0.0001025274743577147, 'epoch': 3.65}
+
 73%|███████▎  | 547/750 [28:22<09:51,  2.91s/it]
 73%|███████▎  | 548/750 [28:24<09:04,  2.70s/it]
                                                 
{'loss': 0.1746, 'grad_norm': 0.2805042266845703, 'learning_rate': 0.00010243388467654152, 'epoch': 3.65}
+
 73%|███████▎  | 548/750 [28:24<09:04,  2.70s/it]
 73%|███████▎  | 549/750 [28:28<09:35,  2.86s/it]
                                                 
{'loss': 0.1611, 'grad_norm': 0.2518482804298401, 'learning_rate': 0.0001023405508214647, 'epoch': 3.66}
+
 73%|███████▎  | 549/750 [28:28<09:35,  2.86s/it]
 73%|███████▎  | 550/750 [28:31<09:42,  2.91s/it]
                                                 
{'loss': 0.2119, 'grad_norm': 0.30590948462486267, 'learning_rate': 0.00010224747162910903, 'epoch': 3.67}
+
 73%|███████▎  | 550/750 [28:31<09:42,  2.91s/it]
 73%|███████▎  | 551/750 [28:33<09:04,  2.73s/it]
                                                 
{'loss': 0.1603, 'grad_norm': 0.24411889910697937, 'learning_rate': 0.00010215464594349242, 'epoch': 3.67}
+
 73%|███████▎  | 551/750 [28:33<09:04,  2.73s/it]
 74%|███████▎  | 552/750 [28:37<10:45,  3.26s/it]
                                                 
{'loss': 0.1735, 'grad_norm': 0.2541138827800751, 'learning_rate': 0.00010206207261596577, 'epoch': 3.68}
+
 74%|███████▎  | 552/750 [28:37<10:45,  3.26s/it]
 74%|███████▎  | 553/750 [28:39<09:25,  2.87s/it]
                                                 
{'loss': 0.1823, 'grad_norm': 0.25020134449005127, 'learning_rate': 0.00010196975050515315, 'epoch': 3.69}
+
 74%|███████▎  | 553/750 [28:39<09:25,  2.87s/it]
 74%|███████▍  | 554/750 [28:42<09:32,  2.92s/it]
                                                 
{'loss': 0.1538, 'grad_norm': 0.27182990312576294, 'learning_rate': 0.00010187767847689286, 'epoch': 3.69}
+
 74%|███████▍  | 554/750 [28:42<09:32,  2.92s/it]
 74%|███████▍  | 555/750 [28:45<09:12,  2.83s/it]
                                                 
{'loss': 0.152, 'grad_norm': 0.24705387651920319, 'learning_rate': 0.00010178585540417863, 'epoch': 3.7}
+
 74%|███████▍  | 555/750 [28:45<09:12,  2.83s/it]
 74%|███████▍  | 556/750 [28:48<09:08,  2.83s/it]
                                                 
{'loss': 0.1708, 'grad_norm': 0.2884485125541687, 'learning_rate': 0.00010169428016710201, 'epoch': 3.71}
+
 74%|███████▍  | 556/750 [28:48<09:08,  2.83s/it]
 74%|███████▍  | 557/750 [28:50<08:36,  2.67s/it]
                                                 
{'loss': 0.1719, 'grad_norm': 0.23405903577804565, 'learning_rate': 0.00010160295165279474, 'epoch': 3.71}
+
 74%|███████▍  | 557/750 [28:50<08:36,  2.67s/it]
 74%|███████▍  | 558/750 [28:53<08:26,  2.64s/it]
                                                 
{'loss': 0.1567, 'grad_norm': 0.2512505352497101, 'learning_rate': 0.00010151186875537208, 'epoch': 3.72}
+
 74%|███████▍  | 558/750 [28:53<08:26,  2.64s/it]
 75%|███████▍  | 559/750 [28:55<08:29,  2.67s/it]
                                                 
{'loss': 0.1874, 'grad_norm': 0.2619980573654175, 'learning_rate': 0.00010142103037587655, 'epoch': 3.73}
+
 75%|███████▍  | 559/750 [28:55<08:29,  2.67s/it]
 75%|███████▍  | 560/750 [28:58<08:00,  2.53s/it]
                                                 
{'loss': 0.1342, 'grad_norm': 0.23432059586048126, 'learning_rate': 0.00010133043542222218, 'epoch': 3.73}
+
 75%|███████▍  | 560/750 [28:58<08:00,  2.53s/it]
 75%|███████▍  | 561/750 [29:00<07:25,  2.36s/it]
                                                 
{'loss': 0.1631, 'grad_norm': 0.2642074525356293, 'learning_rate': 0.00010124008280913954, 'epoch': 3.74}
+
 75%|███████▍  | 561/750 [29:00<07:25,  2.36s/it]
 75%|███████▍  | 562/750 [29:02<07:40,  2.45s/it]
                                                 
{'loss': 0.1424, 'grad_norm': 0.2524045407772064, 'learning_rate': 0.00010114997145812079, 'epoch': 3.75}
+
 75%|███████▍  | 562/750 [29:02<07:40,  2.45s/it]
 75%|███████▌  | 563/750 [29:04<07:24,  2.38s/it]
                                                 
{'loss': 0.1186, 'grad_norm': 0.2400650978088379, 'learning_rate': 0.000101060100297366, 'epoch': 3.75}
+
 75%|███████▌  | 563/750 [29:04<07:24,  2.38s/it]
 75%|███████▌  | 564/750 [29:07<07:42,  2.49s/it]
                                                 
{'loss': 0.1566, 'grad_norm': 0.24889414012432098, 'learning_rate': 0.00010097046826172943, 'epoch': 3.76}
+
 75%|███████▌  | 564/750 [29:07<07:42,  2.49s/it]
 75%|███████▌  | 565/750 [29:10<07:39,  2.48s/it]
                                                 
{'loss': 0.1446, 'grad_norm': 0.24879612028598785, 'learning_rate': 0.0001008810742926664, 'epoch': 3.77}
+
 75%|███████▌  | 565/750 [29:10<07:39,  2.48s/it]
 75%|███████▌  | 566/750 [29:13<08:08,  2.66s/it]
                                                 
{'loss': 0.1627, 'grad_norm': 0.2696031928062439, 'learning_rate': 0.00010079191733818102, 'epoch': 3.77}
+
 75%|███████▌  | 566/750 [29:13<08:08,  2.66s/it]
 76%|███████▌  | 567/750 [29:15<07:39,  2.51s/it]
                                                 
{'loss': 0.1807, 'grad_norm': 0.27792301774024963, 'learning_rate': 0.00010070299635277398, 'epoch': 3.78}
+
 76%|███████▌  | 567/750 [29:15<07:39,  2.51s/it]
 76%|███████▌  | 568/750 [29:18<07:47,  2.57s/it]
                                                 
{'loss': 0.1623, 'grad_norm': 0.26977840065956116, 'learning_rate': 0.00010061431029739118, 'epoch': 3.79}
+
 76%|███████▌  | 568/750 [29:18<07:47,  2.57s/it]
 76%|███████▌  | 569/750 [29:21<08:48,  2.92s/it]
                                                 
{'loss': 0.1529, 'grad_norm': 0.24823182821273804, 'learning_rate': 0.00010052585813937274, 'epoch': 3.79}
+
 76%|███████▌  | 569/750 [29:21<08:48,  2.92s/it]
 76%|███████▌  | 570/750 [29:25<09:12,  3.07s/it]
                                                 
{'loss': 0.1855, 'grad_norm': 0.33246004581451416, 'learning_rate': 0.0001004376388524024, 'epoch': 3.8}
+
 76%|███████▌  | 570/750 [29:25<09:12,  3.07s/it]
 76%|███████▌  | 571/750 [29:27<08:27,  2.83s/it]
                                                 
{'loss': 0.1488, 'grad_norm': 0.25637027621269226, 'learning_rate': 0.00010034965141645762, 'epoch': 3.81}
+
 76%|███████▌  | 571/750 [29:27<08:27,  2.83s/it]
 76%|███████▋  | 572/750 [29:29<07:24,  2.50s/it]
                                                 
{'loss': 0.1886, 'grad_norm': 0.3253023326396942, 'learning_rate': 0.00010026189481775992, 'epoch': 3.81}
+
 76%|███████▋  | 572/750 [29:29<07:24,  2.50s/it]
 76%|███████▋  | 573/750 [29:31<07:26,  2.52s/it]
                                                 
{'loss': 0.1765, 'grad_norm': 0.26307255029678345, 'learning_rate': 0.00010017436804872594, 'epoch': 3.82}
+
 76%|███████▋  | 573/750 [29:31<07:26,  2.52s/it]
 77%|███████▋  | 574/750 [29:34<07:52,  2.68s/it]
                                                 
{'loss': 0.15, 'grad_norm': 0.2940928339958191, 'learning_rate': 0.00010008707010791883, 'epoch': 3.83}
+
 77%|███████▋  | 574/750 [29:34<07:52,  2.68s/it]
 77%|███████▋  | 575/750 [29:37<07:23,  2.54s/it]
                                                 
{'loss': 0.1665, 'grad_norm': 0.24677224457263947, 'learning_rate': 0.0001, 'epoch': 3.83}
+
 77%|███████▋  | 575/750 [29:37<07:23,  2.54s/it]
 77%|███████▋  | 576/750 [29:39<07:09,  2.47s/it]
                                                 
{'loss': 0.146, 'grad_norm': 0.26295506954193115, 'learning_rate': 9.991315673568166e-05, 'epoch': 3.84}
+
 77%|███████▋  | 576/750 [29:39<07:09,  2.47s/it]
 77%|███████▋  | 577/750 [29:41<07:04,  2.46s/it]
                                                 
{'loss': 0.1494, 'grad_norm': 0.2593269348144531, 'learning_rate': 9.982653933167944e-05, 'epoch': 3.85}
+
 77%|███████▋  | 577/750 [29:41<07:04,  2.46s/it]
 77%|███████▋  | 578/750 [29:44<07:06,  2.48s/it]
                                                 
{'loss': 0.1485, 'grad_norm': 0.24497739970684052, 'learning_rate': 9.97401468106657e-05, 'epoch': 3.85}
+
 77%|███████▋  | 578/750 [29:44<07:06,  2.48s/it]
 77%|███████▋  | 579/750 [29:47<07:41,  2.70s/it]
                                                 
{'loss': 0.149, 'grad_norm': 0.22650903463363647, 'learning_rate': 9.965397820122326e-05, 'epoch': 3.86}
+
 77%|███████▋  | 579/750 [29:47<07:41,  2.70s/it]
 77%|███████▋  | 580/750 [29:50<07:29,  2.65s/it]
                                                 
{'loss': 0.1419, 'grad_norm': 0.23806792497634888, 'learning_rate': 9.956803253779939e-05, 'epoch': 3.87}
+
 77%|███████▋  | 580/750 [29:50<07:29,  2.65s/it]
 77%|███████▋  | 581/750 [29:53<08:02,  2.86s/it]
                                                 
{'loss': 0.1702, 'grad_norm': 0.2640670835971832, 'learning_rate': 9.94823088606605e-05, 'epoch': 3.87}
+
 77%|███████▋  | 581/750 [29:53<08:02,  2.86s/it]
 78%|███████▊  | 582/750 [29:56<08:16,  2.95s/it]
                                                 
{'loss': 0.166, 'grad_norm': 0.28246554732322693, 'learning_rate': 9.939680621584706e-05, 'epoch': 3.88}
+
 78%|███████▊  | 582/750 [29:56<08:16,  2.95s/it]
 78%|███████▊  | 583/750 [29:58<07:17,  2.62s/it]
                                                 
{'loss': 0.2043, 'grad_norm': 0.2868112325668335, 'learning_rate': 9.931152365512906e-05, 'epoch': 3.89}
+
 78%|███████▊  | 583/750 [29:58<07:17,  2.62s/it]
 78%|███████▊  | 584/750 [30:01<07:19,  2.65s/it]
                                                 
{'loss': 0.1554, 'grad_norm': 0.23381252586841583, 'learning_rate': 9.922646023596178e-05, 'epoch': 3.89}
+
 78%|███████▊  | 584/750 [30:01<07:19,  2.65s/it]
 78%|███████▊  | 585/750 [30:03<06:49,  2.48s/it]
                                                 
{'loss': 0.1471, 'grad_norm': 0.2667142450809479, 'learning_rate': 9.914161502144209e-05, 'epoch': 3.9}
+
 78%|███████▊  | 585/750 [30:03<06:49,  2.48s/it]
 78%|███████▊  | 586/750 [30:06<07:13,  2.64s/it]
                                                 
{'loss': 0.1781, 'grad_norm': 0.28064584732055664, 'learning_rate': 9.905698708026503e-05, 'epoch': 3.91}
+
 78%|███████▊  | 586/750 [30:06<07:13,  2.64s/it]
 78%|███████▊  | 587/750 [30:09<07:55,  2.91s/it]
                                                 
{'loss': 0.1506, 'grad_norm': 0.24878308176994324, 'learning_rate': 9.897257548668106e-05, 'epoch': 3.91}
+
 78%|███████▊  | 587/750 [30:09<07:55,  2.91s/it]
 78%|███████▊  | 588/750 [30:12<07:37,  2.83s/it]
                                                 
{'loss': 0.1316, 'grad_norm': 0.24419435858726501, 'learning_rate': 9.888837932045328e-05, 'epoch': 3.92}
+
 78%|███████▊  | 588/750 [30:12<07:37,  2.83s/it]
 79%|███████▊  | 589/750 [30:15<07:45,  2.89s/it]
                                                 
{'loss': 0.1941, 'grad_norm': 0.29437607526779175, 'learning_rate': 9.880439766681545e-05, 'epoch': 3.93}
+
 79%|███████▊  | 589/750 [30:15<07:45,  2.89s/it]
 79%|███████▊  | 590/750 [30:17<07:11,  2.70s/it]
                                                 
{'loss': 0.1521, 'grad_norm': 0.2698162794113159, 'learning_rate': 9.872062961643026e-05, 'epoch': 3.93}
+
 79%|███████▊  | 590/750 [30:17<07:11,  2.70s/it]
 79%|███████▉  | 591/750 [30:20<07:18,  2.76s/it]
                                                 
{'loss': 0.1667, 'grad_norm': 0.26913100481033325, 'learning_rate': 9.863707426534786e-05, 'epoch': 3.94}
+
 79%|███████▉  | 591/750 [30:20<07:18,  2.76s/it]
 79%|███████▉  | 592/750 [30:23<07:13,  2.74s/it]
                                                 
{'loss': 0.1799, 'grad_norm': 0.28861743211746216, 'learning_rate': 9.8553730714965e-05, 'epoch': 3.95}
+
 79%|███████▉  | 592/750 [30:23<07:13,  2.74s/it]
 79%|███████▉  | 593/750 [30:26<07:17,  2.79s/it]
                                                 
{'loss': 0.169, 'grad_norm': 0.27587783336639404, 'learning_rate': 9.847059807198442e-05, 'epoch': 3.95}
+
 79%|███████▉  | 593/750 [30:26<07:17,  2.79s/it]
 79%|███████▉  | 594/750 [30:28<07:04,  2.72s/it]
                                                 
{'loss': 0.2007, 'grad_norm': 0.27922022342681885, 'learning_rate': 9.838767544837452e-05, 'epoch': 3.96}
+
 79%|███████▉  | 594/750 [30:28<07:04,  2.72s/it]
 79%|███████▉  | 595/750 [30:30<06:22,  2.47s/it]
                                                 
{'loss': 0.1819, 'grad_norm': 0.31566137075424194, 'learning_rate': 9.830496196132975e-05, 'epoch': 3.97}
+
 79%|███████▉  | 595/750 [30:30<06:22,  2.47s/it]
 79%|███████▉  | 596/750 [30:33<06:40,  2.60s/it]
                                                 
{'loss': 0.1519, 'grad_norm': 0.26233938336372375, 'learning_rate': 9.822245673323086e-05, 'epoch': 3.97}
+
 79%|███████▉  | 596/750 [30:33<06:40,  2.60s/it]
 80%|███████▉  | 597/750 [30:36<06:36,  2.59s/it]
                                                 
{'loss': 0.1577, 'grad_norm': 0.2438373565673828, 'learning_rate': 9.814015889160605e-05, 'epoch': 3.98}
+
 80%|███████▉  | 597/750 [30:36<06:36,  2.59s/it]
 80%|███████▉  | 598/750 [30:38<06:42,  2.65s/it]
                                                 
{'loss': 0.1724, 'grad_norm': 0.2693221867084503, 'learning_rate': 9.805806756909202e-05, 'epoch': 3.99}
+
 80%|███████▉  | 598/750 [30:38<06:42,  2.65s/it]
 80%|███████▉  | 599/750 [30:41<06:55,  2.75s/it]
                                                 
{'loss': 0.1491, 'grad_norm': 0.251712441444397, 'learning_rate': 9.797618190339569e-05, 'epoch': 3.99}
+
 80%|███████▉  | 599/750 [30:41<06:55,  2.75s/it]
 80%|████████  | 600/750 [30:46<08:18,  3.32s/it]
                                                 
{'loss': 0.1374, 'grad_norm': 0.23223747313022614, 'learning_rate': 9.789450103725609e-05, 'epoch': 4.0}
+
 80%|████████  | 600/750 [30:46<08:18,  3.32s/it][INFO|trainer.py:3831] 2025-06-27 01:39:24,472 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-27 01:39:24,472 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-27 01:39:24,472 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:00<00:03,  2.43it/s]
+
 30%|███       | 3/10 [00:01<00:03,  1.75it/s]
+
 40%|████      | 4/10 [00:02<00:03,  1.62it/s]
+
 50%|█████     | 5/10 [00:03<00:04,  1.20it/s]
+
 60%|██████    | 6/10 [00:04<00:03,  1.21it/s]
+
 70%|███████   | 7/10 [00:05<00:02,  1.29it/s]
+
 80%|████████  | 8/10 [00:05<00:01,  1.41it/s]
+
 90%|█████████ | 9/10 [00:06<00:00,  1.30it/s]
+
100%|██████████| 10/10 [00:09<00:00,  1.35s/it]
                                                 
+
                                               
{'eval_loss': 0.3029455542564392, 'eval_accuracy': 0.46136664460622107, 'eval_runtime': 22.3286, 'eval_samples_per_second': 44.786, 'eval_steps_per_second': 0.448, 'epoch': 4.0}
+
 80%|████████  | 600/750 [31:08<08:18,  3.32s/it]
+
100%|██████████| 10/10 [00:13<00:00,  1.35s/it]
+
                                               /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[INFO|trainer.py:3515] 2025-06-27 01:40:00,334 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-600
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da918-6b5425e3186dcf3844e1145d;28e085c5-38fc-4070-86f3-552d274090d7)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-27 01:40:01,070 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-600/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-27 01:40:01,070 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-600/special_tokens_map.json
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[2025-06-27 01:40:02,102] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Checkpoint global_step600 is begin to save!
+[2025-06-27 01:40:02,125] [INFO] [logging.py:107:log_dist] [Rank 0] Saving model checkpoint: ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-600/global_step600/mp_rank_00_model_states.pt
+[INFO|trainer.py:3607] 2025-06-27 01:40:02,243 >> Deleting older checkpoint [outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-400] due to args.save_total_limit
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da91a-035ff7a938f9405023e8ea15;758c5edf-7a84-4fb8-9ebc-539e95a6af34)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da91a-5a4701d5205b665743599c3c;b9deb804-8661-4c98-b12a-77c5650cd3bd)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da91a-35d842ce1aacf33c721b0e90;6c10d5d8-652d-4a64-bc7b-d8617ae91260)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685da91a-7a071f1a198815957e71a171;1a859454-591e-4810-a119-5014f8d14ddb)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+
 80%|████████  | 601/750 [31:34<41:45, 16.82s/it]
                                                 
{'loss': 0.1419, 'grad_norm': 0.26574763655662537, 'learning_rate': 9.781302411840674e-05, 'epoch': 4.01}
+
 80%|████████  | 601/750 [31:34<41:45, 16.82s/it]
 80%|████████  | 602/750 [31:37<30:54, 12.53s/it]
                                                 
{'loss': 0.1237, 'grad_norm': 0.2460770308971405, 'learning_rate': 9.773175029953825e-05, 'epoch': 4.01}
+
 80%|████████  | 602/750 [31:37<30:54, 12.53s/it]
 80%|████████  | 603/750 [31:39<23:04,  9.42s/it]
                                                 
{'loss': 0.1048, 'grad_norm': 0.24162235856056213, 'learning_rate': 9.76506787382613e-05, 'epoch': 4.02}
+
 80%|████████  | 603/750 [31:39<23:04,  9.42s/it]
 81%|████████  | 604/750 [31:42<18:20,  7.53s/it]
                                                 
{'loss': 0.1205, 'grad_norm': 0.26170578598976135, 'learning_rate': 9.756980859707e-05, 'epoch': 4.03}
+
 81%|████████  | 604/750 [31:42<18:20,  7.53s/it]
 81%|████████  | 605/750 [31:47<15:51,  6.56s/it]
                                                 
{'loss': 0.1107, 'grad_norm': 0.2399194985628128, 'learning_rate': 9.748913904330553e-05, 'epoch': 4.03}
+
 81%|████████  | 605/750 [31:47<15:51,  6.56s/it]
 81%|████████  | 606/750 [31:49<13:00,  5.42s/it]
                                                 
{'loss': 0.1025, 'grad_norm': 0.2631273567676544, 'learning_rate': 9.740866924912017e-05, 'epoch': 4.04}
+
 81%|████████  | 606/750 [31:49<13:00,  5.42s/it]
 81%|████████  | 607/750 [31:52<10:55,  4.58s/it]
                                                 
{'loss': 0.1146, 'grad_norm': 0.3020670413970947, 'learning_rate': 9.732839839144154e-05, 'epoch': 4.05}
+
 81%|████████  | 607/750 [31:52<10:55,  4.58s/it]
 81%|████████  | 608/750 [31:55<09:37,  4.07s/it]
                                                 
{'loss': 0.1177, 'grad_norm': 0.38536593317985535, 'learning_rate': 9.724832565193738e-05, 'epoch': 4.05}
+
 81%|████████  | 608/750 [31:55<09:37,  4.07s/it]
 81%|████████  | 609/750 [31:57<08:34,  3.65s/it]
                                                 
{'loss': 0.1197, 'grad_norm': 0.36749619245529175, 'learning_rate': 9.716845021698033e-05, 'epoch': 4.06}
+
 81%|████████  | 609/750 [31:57<08:34,  3.65s/it]
 81%|████████▏ | 610/750 [32:01<08:18,  3.56s/it]
                                                 
{'loss': 0.1042, 'grad_norm': 0.30136269330978394, 'learning_rate': 9.708877127761337e-05, 'epoch': 4.07}
+
 81%|████████▏ | 610/750 [32:01<08:18,  3.56s/it]
 81%|████████▏ | 611/750 [32:04<08:11,  3.54s/it]
                                                 
{'loss': 0.1249, 'grad_norm': 0.3005106449127197, 'learning_rate': 9.700928802951527e-05, 'epoch': 4.07}
+
 81%|████████▏ | 611/750 [32:04<08:11,  3.54s/it]
 82%|████████▏ | 612/750 [32:07<07:26,  3.23s/it]
                                                 
{'loss': 0.1152, 'grad_norm': 0.35263633728027344, 'learning_rate': 9.69299996729666e-05, 'epoch': 4.08}
+
 82%|████████▏ | 612/750 [32:07<07:26,  3.23s/it]
 82%|████████▏ | 613/750 [32:10<07:08,  3.13s/it]
                                                 
{'loss': 0.1148, 'grad_norm': 0.2801554203033447, 'learning_rate': 9.685090541281594e-05, 'epoch': 4.09}
+
 82%|████████▏ | 613/750 [32:10<07:08,  3.13s/it]
 82%|████████▏ | 614/750 [32:13<06:54,  3.05s/it]
                                                 
{'loss': 0.0889, 'grad_norm': 0.25111910700798035, 'learning_rate': 9.677200445844624e-05, 'epoch': 4.09}
+
 82%|████████▏ | 614/750 [32:13<06:54,  3.05s/it]
 82%|████████▏ | 615/750 [32:15<06:37,  2.94s/it]
                                                 
{'loss': 0.1244, 'grad_norm': 0.293525755405426, 'learning_rate': 9.669329602374179e-05, 'epoch': 4.1}
+
 82%|████████▏ | 615/750 [32:15<06:37,  2.94s/it]
 82%|████████▏ | 616/750 [32:18<06:34,  2.94s/it]
                                                 
{'loss': 0.1285, 'grad_norm': 0.2808990776538849, 'learning_rate': 9.661477932705526e-05, 'epoch': 4.11}
+
 82%|████████▏ | 616/750 [32:18<06:34,  2.94s/it]
 82%|████████▏ | 617/750 [32:22<06:59,  3.15s/it]
                                                 
{'loss': 0.1491, 'grad_norm': 0.31453365087509155, 'learning_rate': 9.65364535911751e-05, 'epoch': 4.11}
+
 82%|████████▏ | 617/750 [32:22<06:59,  3.15s/it]
 82%|████████▏ | 618/750 [32:25<06:50,  3.11s/it]
                                                 
{'loss': 0.1098, 'grad_norm': 0.2967071235179901, 'learning_rate': 9.645831804329319e-05, 'epoch': 4.12}
+
 82%|████████▏ | 618/750 [32:25<06:50,  3.11s/it]
 83%|████████▎ | 619/750 [32:28<06:34,  3.01s/it]
                                                 
{'loss': 0.1159, 'grad_norm': 0.2969232201576233, 'learning_rate': 9.63803719149729e-05, 'epoch': 4.13}
+
 83%|████████▎ | 619/750 [32:28<06:34,  3.01s/it]
 83%|████████▎ | 620/750 [32:30<05:56,  2.74s/it]
                                                 
{'loss': 0.1128, 'grad_norm': 0.29954075813293457, 'learning_rate': 9.630261444211729e-05, 'epoch': 4.13}
+
 83%|████████▎ | 620/750 [32:30<05:56,  2.74s/it]
 83%|████████▎ | 621/750 [32:33<06:21,  2.96s/it]
                                                 
{'loss': 0.1322, 'grad_norm': 0.3027983605861664, 'learning_rate': 9.622504486493763e-05, 'epoch': 4.14}
+
 83%|████████▎ | 621/750 [32:33<06:21,  2.96s/it]
 83%|████████▎ | 622/750 [32:35<05:37,  2.64s/it]
                                                 
{'loss': 0.1131, 'grad_norm': 0.30854764580726624, 'learning_rate': 9.614766242792228e-05, 'epoch': 4.15}
+
 83%|████████▎ | 622/750 [32:35<05:37,  2.64s/it]
 83%|████████▎ | 623/750 [32:37<05:14,  2.47s/it]
                                                 
{'loss': 0.1362, 'grad_norm': 0.3283732831478119, 'learning_rate': 9.607046637980575e-05, 'epoch': 4.15}
+
 83%|████████▎ | 623/750 [32:37<05:14,  2.47s/it]
 83%|████████▎ | 624/750 [32:40<05:09,  2.45s/it]
                                                 
{'loss': 0.1016, 'grad_norm': 0.3071095049381256, 'learning_rate': 9.599345597353806e-05, 'epoch': 4.16}
+
 83%|████████▎ | 624/750 [32:40<05:09,  2.45s/it]
 83%|████████▎ | 625/750 [32:43<05:30,  2.64s/it]
                                                 
{'loss': 0.0952, 'grad_norm': 0.29622113704681396, 'learning_rate': 9.59166304662544e-05, 'epoch': 4.17}
+
 83%|████████▎ | 625/750 [32:43<05:30,  2.64s/it]
 83%|████████▎ | 626/750 [32:46<05:55,  2.87s/it]
                                                 
{'loss': 0.1389, 'grad_norm': 0.3278207778930664, 'learning_rate': 9.5839989119245e-05, 'epoch': 4.17}
+
 83%|████████▎ | 626/750 [32:46<05:55,  2.87s/it]
 84%|████████▎ | 627/750 [32:48<05:30,  2.69s/it]
                                                 
{'loss': 0.1149, 'grad_norm': 0.31515759229660034, 'learning_rate': 9.57635311979253e-05, 'epoch': 4.18}
+
 84%|████████▎ | 627/750 [32:48<05:30,  2.69s/it]
 84%|████████▎ | 628/750 [32:51<05:42,  2.81s/it]
                                                 
{'loss': 0.114, 'grad_norm': 0.3289165794849396, 'learning_rate': 9.568725597180648e-05, 'epoch': 4.19}
+
 84%|████████▎ | 628/750 [32:51<05:42,  2.81s/it]
 84%|████████▍ | 629/750 [32:54<05:17,  2.63s/it]
                                                 
{'loss': 0.0853, 'grad_norm': 0.25508517026901245, 'learning_rate': 9.561116271446594e-05, 'epoch': 4.19}
+
 84%|████████▍ | 629/750 [32:54<05:17,  2.63s/it]
 84%|████████▍ | 630/750 [32:56<05:21,  2.68s/it]
                                                 
{'loss': 0.0989, 'grad_norm': 0.28110384941101074, 'learning_rate': 9.553525070351848e-05, 'epoch': 4.2}
+
 84%|████████▍ | 630/750 [32:56<05:21,  2.68s/it]
 84%|████████▍ | 631/750 [32:59<05:15,  2.65s/it]
                                                 
{'loss': 0.0989, 'grad_norm': 0.30223047733306885, 'learning_rate': 9.54595192205873e-05, 'epoch': 4.21}
+
 84%|████████▍ | 631/750 [32:59<05:15,  2.65s/it]
 84%|████████▍ | 632/750 [33:01<04:56,  2.51s/it]
                                                 
{'loss': 0.1239, 'grad_norm': 0.32768064737319946, 'learning_rate': 9.538396755127553e-05, 'epoch': 4.21}
+
 84%|████████▍ | 632/750 [33:01<04:56,  2.51s/it]
 84%|████████▍ | 633/750 [33:04<04:53,  2.51s/it]
                                                 
{'loss': 0.1302, 'grad_norm': 0.3742484152317047, 'learning_rate': 9.530859498513802e-05, 'epoch': 4.22}
+
 84%|████████▍ | 633/750 [33:04<04:53,  2.51s/it]
 85%|████████▍ | 634/750 [33:06<04:56,  2.56s/it]
                                                 
{'loss': 0.1053, 'grad_norm': 0.35667893290519714, 'learning_rate': 9.523340081565305e-05, 'epoch': 4.23}
+
 85%|████████▍ | 634/750 [33:06<04:56,  2.56s/it]
 85%|████████▍ | 635/750 [33:10<05:34,  2.91s/it]
                                                 
{'loss': 0.1147, 'grad_norm': 0.3369566798210144, 'learning_rate': 9.515838434019473e-05, 'epoch': 4.23}
+
 85%|████████▍ | 635/750 [33:10<05:34,  2.91s/it]
 85%|████████▍ | 636/750 [33:16<07:02,  3.70s/it]
                                                 
{'loss': 0.0953, 'grad_norm': 0.32306107878685, 'learning_rate': 9.508354486000538e-05, 'epoch': 4.24}
+
 85%|████████▍ | 636/750 [33:16<07:02,  3.70s/it]
 85%|████████▍ | 637/750 [33:21<08:07,  4.32s/it]
                                                 
{'loss': 0.0928, 'grad_norm': 0.29875749349594116, 'learning_rate': 9.500888168016802e-05, 'epoch': 4.25}
+
 85%|████████▍ | 637/750 [33:21<08:07,  4.32s/it]
 85%|████████▌ | 638/750 [33:24<06:57,  3.73s/it]
                                                 
{'loss': 0.1187, 'grad_norm': 0.3257039487361908, 'learning_rate': 9.493439410957959e-05, 'epoch': 4.25}
+
 85%|████████▌ | 638/750 [33:24<06:57,  3.73s/it]
 85%|████████▌ | 639/750 [33:29<07:54,  4.28s/it]
                                                 
{'loss': 0.1392, 'grad_norm': 0.35467764735221863, 'learning_rate': 9.48600814609237e-05, 'epoch': 4.26}
+
 85%|████████▌ | 639/750 [33:29<07:54,  4.28s/it]
 85%|████████▌ | 640/750 [33:32<06:48,  3.71s/it]
                                                 
{'loss': 0.1078, 'grad_norm': 0.28514915704727173, 'learning_rate': 9.478594305064439e-05, 'epoch': 4.27}
+
 85%|████████▌ | 640/750 [33:32<06:48,  3.71s/it]
 85%|████████▌ | 641/750 [33:34<06:01,  3.31s/it]
                                                 
{'loss': 0.1257, 'grad_norm': 0.31021982431411743, 'learning_rate': 9.471197819891936e-05, 'epoch': 4.27}
+
 85%|████████▌ | 641/750 [33:34<06:01,  3.31s/it]
 86%|████████▌ | 642/750 [33:36<05:23,  3.00s/it]
                                                 
{'loss': 0.1566, 'grad_norm': 0.3631383180618286, 'learning_rate': 9.463818622963402e-05, 'epoch': 4.28}
+
 86%|████████▌ | 642/750 [33:36<05:23,  3.00s/it]
 86%|████████▌ | 643/750 [33:39<05:21,  3.01s/it]
                                                 
{'loss': 0.1157, 'grad_norm': 0.31213879585266113, 'learning_rate': 9.45645664703555e-05, 'epoch': 4.29}
+
 86%|████████▌ | 643/750 [33:39<05:21,  3.01s/it]
 86%|████████▌ | 644/750 [33:43<05:30,  3.12s/it]
                                                 
{'loss': 0.112, 'grad_norm': 0.3037095069885254, 'learning_rate': 9.44911182523068e-05, 'epoch': 4.29}
+
 86%|████████▌ | 644/750 [33:43<05:30,  3.12s/it]
 86%|████████▌ | 645/750 [33:46<05:20,  3.05s/it]
                                                 
{'loss': 0.0984, 'grad_norm': 0.2882070541381836, 'learning_rate': 9.441784091034141e-05, 'epoch': 4.3}
+
 86%|████████▌ | 645/750 [33:46<05:20,  3.05s/it]
 86%|████████▌ | 646/750 [33:50<05:55,  3.41s/it]
                                                 
{'loss': 0.0999, 'grad_norm': 0.304634153842926, 'learning_rate': 9.434473378291795e-05, 'epoch': 4.31}
+
 86%|████████▌ | 646/750 [33:50<05:55,  3.41s/it]
 86%|████████▋ | 647/750 [33:52<05:23,  3.14s/it]
                                                 
{'loss': 0.1348, 'grad_norm': 0.3070700764656067, 'learning_rate': 9.427179621207505e-05, 'epoch': 4.31}
+
 86%|████████▋ | 647/750 [33:52<05:23,  3.14s/it]
 86%|████████▋ | 648/750 [33:55<05:09,  3.03s/it]
                                                 
{'loss': 0.1323, 'grad_norm': 0.3337042033672333, 'learning_rate': 9.419902754340649e-05, 'epoch': 4.32}
+
 86%|████████▋ | 648/750 [33:55<05:09,  3.03s/it]
 87%|████████▋ | 649/750 [33:59<05:33,  3.31s/it]
                                                 
{'loss': 0.1188, 'grad_norm': 0.3084576725959778, 'learning_rate': 9.41264271260367e-05, 'epoch': 4.33}
+
 87%|████████▋ | 649/750 [33:59<05:33,  3.31s/it]
 87%|████████▋ | 650/750 [34:01<04:48,  2.88s/it]
                                                 
{'loss': 0.1091, 'grad_norm': 0.3431152403354645, 'learning_rate': 9.405399431259603e-05, 'epoch': 4.33}
+
 87%|████████▋ | 650/750 [34:01<04:48,  2.88s/it]
 87%|████████▋ | 651/750 [34:04<04:41,  2.84s/it]
                                                 
{'loss': 0.1254, 'grad_norm': 0.3138129711151123, 'learning_rate': 9.398172845919671e-05, 'epoch': 4.34}
+
 87%|████████▋ | 651/750 [34:04<04:41,  2.84s/it]
 87%|████████▋ | 652/750 [34:07<04:49,  2.96s/it]
                                                 
{'loss': 0.1253, 'grad_norm': 0.3130810260772705, 'learning_rate': 9.390962892540881e-05, 'epoch': 4.35}
+
 87%|████████▋ | 652/750 [34:07<04:49,  2.96s/it]
 87%|████████▋ | 653/750 [34:10<04:42,  2.91s/it]
                                                 
{'loss': 0.1156, 'grad_norm': 0.33594855666160583, 'learning_rate': 9.383769507423618e-05, 'epoch': 4.35}
+
 87%|████████▋ | 653/750 [34:10<04:42,  2.91s/it]
 87%|████████▋ | 654/750 [34:12<04:31,  2.82s/it]
                                                 
{'loss': 0.1242, 'grad_norm': 0.32983097434043884, 'learning_rate': 9.376592627209315e-05, 'epoch': 4.36}
+
 87%|████████▋ | 654/750 [34:13<04:31,  2.82s/it]
 87%|████████▋ | 655/750 [34:15<04:30,  2.85s/it]
                                                 
{'loss': 0.0936, 'grad_norm': 0.3034319579601288, 'learning_rate': 9.369432188878082e-05, 'epoch': 4.37}
+
 87%|████████▋ | 655/750 [34:15<04:30,  2.85s/it]
 87%|████████▋ | 656/750 [34:18<04:19,  2.76s/it]
                                                 
{'loss': 0.1194, 'grad_norm': 0.34263646602630615, 'learning_rate': 9.362288129746395e-05, 'epoch': 4.37}
+
 87%|████████▋ | 656/750 [34:18<04:19,  2.76s/it]
 88%|████████▊ | 657/750 [34:21<04:31,  2.91s/it]
                                                 
{'loss': 0.1007, 'grad_norm': 0.26024526357650757, 'learning_rate': 9.355160387464784e-05, 'epoch': 4.38}
+
 88%|████████▊ | 657/750 [34:21<04:31,  2.91s/it]
 88%|████████▊ | 658/750 [34:24<04:33,  2.97s/it]
                                                 
{'loss': 0.1102, 'grad_norm': 0.28676143288612366, 'learning_rate': 9.348048900015556e-05, 'epoch': 4.39}
+
 88%|████████▊ | 658/750 [34:24<04:33,  2.97s/it]
 88%|████████▊ | 659/750 [34:27<04:29,  2.97s/it]
                                                 
{'loss': 0.0972, 'grad_norm': 0.2715829610824585, 'learning_rate': 9.340953605710524e-05, 'epoch': 4.39}
+
 88%|████████▊ | 659/750 [34:27<04:29,  2.97s/it]
 88%|████████▊ | 660/750 [34:30<04:10,  2.78s/it]
                                                 
{'loss': 0.1211, 'grad_norm': 0.32494691014289856, 'learning_rate': 9.333874443188752e-05, 'epoch': 4.4}
+
 88%|████████▊ | 660/750 [34:30<04:10,  2.78s/it]
 88%|████████▊ | 661/750 [34:32<04:07,  2.78s/it]
                                                 
{'loss': 0.1358, 'grad_norm': 0.4287351071834564, 'learning_rate': 9.326811351414343e-05, 'epoch': 4.41}
+
 88%|████████▊ | 661/750 [34:32<04:07,  2.78s/it]
 88%|████████▊ | 662/750 [34:35<03:58,  2.71s/it]
                                                 
{'loss': 0.1021, 'grad_norm': 0.30165454745292664, 'learning_rate': 9.319764269674211e-05, 'epoch': 4.41}
+
 88%|████████▊ | 662/750 [34:35<03:58,  2.71s/it]
 88%|████████▊ | 663/750 [34:38<03:57,  2.73s/it]
                                                 
{'loss': 0.1382, 'grad_norm': 0.304910808801651, 'learning_rate': 9.312733137575907e-05, 'epoch': 4.42}
+
 88%|████████▊ | 663/750 [34:38<03:57,  2.73s/it]
 89%|████████▊ | 664/750 [34:41<03:59,  2.78s/it]
                                                 
{'loss': 0.1326, 'grad_norm': 0.3049857020378113, 'learning_rate': 9.305717895045427e-05, 'epoch': 4.43}
+
 89%|████████▊ | 664/750 [34:41<03:59,  2.78s/it]
 89%|████████▊ | 665/750 [34:43<03:34,  2.52s/it]
                                                 
{'loss': 0.1363, 'grad_norm': 0.3137211501598358, 'learning_rate': 9.298718482325068e-05, 'epoch': 4.43}
+
 89%|████████▊ | 665/750 [34:43<03:34,  2.52s/it]
 89%|████████▉ | 666/750 [34:45<03:35,  2.57s/it]
                                                 
{'loss': 0.1223, 'grad_norm': 0.3528100550174713, 'learning_rate': 9.291734839971293e-05, 'epoch': 4.44}
+
 89%|████████▉ | 666/750 [34:45<03:35,  2.57s/it]
 89%|████████▉ | 667/750 [34:48<03:41,  2.67s/it]
                                                 
{'loss': 0.1262, 'grad_norm': 0.3025120198726654, 'learning_rate': 9.284766908852593e-05, 'epoch': 4.45}
+
 89%|████████▉ | 667/750 [34:48<03:41,  2.67s/it]
 89%|████████▉ | 668/750 [34:51<03:32,  2.59s/it]
                                                 
{'loss': 0.1077, 'grad_norm': 0.30296942591667175, 'learning_rate': 9.277814630147407e-05, 'epoch': 4.45}
+
 89%|████████▉ | 668/750 [34:51<03:32,  2.59s/it]
 89%|████████▉ | 669/750 [34:53<03:23,  2.51s/it]
                                                 
{'loss': 0.1326, 'grad_norm': 0.3140397071838379, 'learning_rate': 9.270877945342015e-05, 'epoch': 4.46}
+
 89%|████████▉ | 669/750 [34:53<03:23,  2.51s/it]
 89%|████████▉ | 670/750 [34:55<03:18,  2.48s/it]
                                                 
{'loss': 0.1035, 'grad_norm': 0.29305315017700195, 'learning_rate': 9.263956796228493e-05, 'epoch': 4.47}
+
 89%|████████▉ | 670/750 [34:55<03:18,  2.48s/it]
 89%|████████▉ | 671/750 [34:57<03:05,  2.35s/it]
                                                 
{'loss': 0.1046, 'grad_norm': 0.3158821165561676, 'learning_rate': 9.257051124902634e-05, 'epoch': 4.47}
+
 89%|████████▉ | 671/750 [34:57<03:05,  2.35s/it]
 90%|████████▉ | 672/750 [35:00<03:17,  2.54s/it]
                                                 
{'loss': 0.1223, 'grad_norm': 0.36178553104400635, 'learning_rate': 9.250160873761937e-05, 'epoch': 4.48}
+
 90%|████████▉ | 672/750 [35:00<03:17,  2.54s/it]
 90%|████████▉ | 673/750 [35:02<03:05,  2.41s/it]
                                                 
{'loss': 0.0989, 'grad_norm': 0.2870869040489197, 'learning_rate': 9.243285985503578e-05, 'epoch': 4.49}
+
 90%|████████▉ | 673/750 [35:02<03:05,  2.41s/it]
 90%|████████▉ | 674/750 [35:05<02:57,  2.33s/it]
                                                 
{'loss': 0.1109, 'grad_norm': 0.2921835780143738, 'learning_rate': 9.236426403122409e-05, 'epoch': 4.49}
+
 90%|████████▉ | 674/750 [35:05<02:57,  2.33s/it]
 90%|█████████ | 675/750 [35:08<03:12,  2.56s/it]
                                                 
{'loss': 0.1093, 'grad_norm': 0.3153856098651886, 'learning_rate': 9.229582069908974e-05, 'epoch': 4.5}
+
 90%|█████████ | 675/750 [35:08<03:12,  2.56s/it]
 90%|█████████ | 676/750 [35:10<03:13,  2.61s/it]
                                                 
{'loss': 0.1378, 'grad_norm': 0.34532415866851807, 'learning_rate': 9.222752929447538e-05, 'epoch': 4.51}
+
 90%|█████████ | 676/750 [35:10<03:13,  2.61s/it]
 90%|█████████ | 677/750 [35:12<02:55,  2.41s/it]
                                                 
{'loss': 0.1104, 'grad_norm': 0.29405471682548523, 'learning_rate': 9.215938925614141e-05, 'epoch': 4.51}
+
 90%|█████████ | 677/750 [35:12<02:55,  2.41s/it]
 90%|█████████ | 678/750 [35:15<03:03,  2.55s/it]
                                                 
{'loss': 0.0953, 'grad_norm': 0.2816740870475769, 'learning_rate': 9.209140002574652e-05, 'epoch': 4.52}
+
 90%|█████████ | 678/750 [35:15<03:03,  2.55s/it]
 91%|█████████ | 679/750 [35:19<03:26,  2.91s/it]
                                                 
{'loss': 0.1297, 'grad_norm': 0.3373907208442688, 'learning_rate': 9.20235610478285e-05, 'epoch': 4.53}
+
 91%|█████████ | 679/750 [35:19<03:26,  2.91s/it]
 91%|█████████ | 680/750 [35:22<03:18,  2.84s/it]
                                                 
{'loss': 0.1135, 'grad_norm': 0.30661481618881226, 'learning_rate': 9.19558717697852e-05, 'epoch': 4.53}
+
 91%|█████████ | 680/750 [35:22<03:18,  2.84s/it]
 91%|█████████ | 681/750 [35:24<02:57,  2.58s/it]
                                                 
{'loss': 0.1173, 'grad_norm': 0.3302581012248993, 'learning_rate': 9.18883316418556e-05, 'epoch': 4.54}
+
 91%|█████████ | 681/750 [35:24<02:57,  2.58s/it]
 91%|█████████ | 682/750 [35:26<02:45,  2.43s/it]
                                                 
{'loss': 0.1158, 'grad_norm': 0.32692116498947144, 'learning_rate': 9.182094011710111e-05, 'epoch': 4.55}
+
 91%|█████████ | 682/750 [35:26<02:45,  2.43s/it]
 91%|█████████ | 683/750 [35:28<02:33,  2.29s/it]
                                                 
{'loss': 0.1263, 'grad_norm': 0.2900867760181427, 'learning_rate': 9.175369665138691e-05, 'epoch': 4.55}
+
 91%|█████████ | 683/750 [35:28<02:33,  2.29s/it]
 91%|█████████ | 684/750 [35:31<02:51,  2.60s/it]
                                                 
{'loss': 0.1094, 'grad_norm': 0.28536152839660645, 'learning_rate': 9.168660070336348e-05, 'epoch': 4.56}
+
 91%|█████████ | 684/750 [35:31<02:51,  2.60s/it]
 91%|█████████▏| 685/750 [35:34<02:50,  2.63s/it]
                                                 
{'loss': 0.111, 'grad_norm': 0.29989543557167053, 'learning_rate': 9.161965173444834e-05, 'epoch': 4.57}
+
 91%|█████████▏| 685/750 [35:34<02:50,  2.63s/it]
 91%|█████████▏| 686/750 [35:36<02:42,  2.54s/it]
                                                 
{'loss': 0.1302, 'grad_norm': 0.350248247385025, 'learning_rate': 9.155284920880796e-05, 'epoch': 4.57}
+
 91%|█████████▏| 686/750 [35:36<02:42,  2.54s/it]
 92%|█████████▏| 687/750 [35:39<02:45,  2.62s/it]
                                                 
{'loss': 0.0941, 'grad_norm': 0.2537909150123596, 'learning_rate': 9.148619259333955e-05, 'epoch': 4.58}
+
 92%|█████████▏| 687/750 [35:39<02:45,  2.62s/it]
 92%|█████████▏| 688/750 [35:42<02:50,  2.75s/it]
                                                 
{'loss': 0.1125, 'grad_norm': 0.3028033673763275, 'learning_rate': 9.141968135765342e-05, 'epoch': 4.59}
+
 92%|█████████▏| 688/750 [35:42<02:50,  2.75s/it]
 92%|█████████▏| 689/750 [35:45<02:59,  2.93s/it]
                                                 
{'loss': 0.1005, 'grad_norm': 0.2873552739620209, 'learning_rate': 9.135331497405505e-05, 'epoch': 4.59}
+
 92%|█████████▏| 689/750 [35:45<02:59,  2.93s/it]
 92%|█████████▏| 690/750 [35:48<02:58,  2.97s/it]
                                                 
{'loss': 0.1189, 'grad_norm': 0.31909283995628357, 'learning_rate': 9.128709291752768e-05, 'epoch': 4.6}
+
 92%|█████████▏| 690/750 [35:48<02:58,  2.97s/it]
 92%|█████████▏| 691/750 [35:51<02:48,  2.85s/it]
                                                 
{'loss': 0.1416, 'grad_norm': 0.3461187183856964, 'learning_rate': 9.122101466571477e-05, 'epoch': 4.61}
+
 92%|█████████▏| 691/750 [35:51<02:48,  2.85s/it]
 92%|█████████▏| 692/750 [35:54<02:54,  3.01s/it]
                                                 
{'loss': 0.1112, 'grad_norm': 0.295687735080719, 'learning_rate': 9.115507969890265e-05, 'epoch': 4.61}
+
 92%|█████████▏| 692/750 [35:54<02:54,  3.01s/it]
 92%|█████████▏| 693/750 [35:57<02:48,  2.96s/it]
                                                 
{'loss': 0.1253, 'grad_norm': 0.3461240828037262, 'learning_rate': 9.108928750000352e-05, 'epoch': 4.62}
+
 92%|█████████▏| 693/750 [35:57<02:48,  2.96s/it]
 93%|█████████▎| 694/750 [36:00<02:52,  3.09s/it]
                                                 
{'loss': 0.1372, 'grad_norm': 0.3381611704826355, 'learning_rate': 9.102363755453824e-05, 'epoch': 4.63}
+
 93%|█████████▎| 694/750 [36:00<02:52,  3.09s/it]
 93%|█████████▎| 695/750 [36:03<02:43,  2.98s/it]
                                                 
{'loss': 0.0999, 'grad_norm': 0.28714969754219055, 'learning_rate': 9.095812935061951e-05, 'epoch': 4.63}
+
 93%|█████████▎| 695/750 [36:03<02:43,  2.98s/it]
 93%|█████████▎| 696/750 [36:06<02:44,  3.04s/it]
                                                 
{'loss': 0.121, 'grad_norm': 0.3212418854236603, 'learning_rate': 9.089276237893512e-05, 'epoch': 4.64}
+
 93%|█████████▎| 696/750 [36:06<02:44,  3.04s/it]
 93%|█████████▎| 697/750 [36:10<02:43,  3.09s/it]
                                                 
{'loss': 0.117, 'grad_norm': 0.3266619145870209, 'learning_rate': 9.082753613273131e-05, 'epoch': 4.65}
+
 93%|█████████▎| 697/750 [36:10<02:43,  3.09s/it]
 93%|█████████▎| 698/750 [36:12<02:26,  2.82s/it]
                                                 
{'loss': 0.1194, 'grad_norm': 0.31951677799224854, 'learning_rate': 9.076245010779625e-05, 'epoch': 4.65}
+
 93%|█████████▎| 698/750 [36:12<02:26,  2.82s/it]
 93%|█████████▎| 699/750 [36:16<02:40,  3.14s/it]
                                                 
{'loss': 0.0918, 'grad_norm': 0.3154314458370209, 'learning_rate': 9.069750380244364e-05, 'epoch': 4.66}
+
 93%|█████████▎| 699/750 [36:16<02:40,  3.14s/it]
 93%|█████████▎| 700/750 [36:18<02:17,  2.75s/it]
                                                 
{'loss': 0.111, 'grad_norm': 0.3571970462799072, 'learning_rate': 9.063269671749657e-05, 'epoch': 4.67}
+
 93%|█████████▎| 700/750 [36:18<02:17,  2.75s/it]
 93%|█████████▎| 701/750 [36:21<02:21,  2.88s/it]
                                                 
{'loss': 0.1036, 'grad_norm': 0.2859587073326111, 'learning_rate': 9.056802835627134e-05, 'epoch': 4.67}
+
 93%|█████████▎| 701/750 [36:21<02:21,  2.88s/it]
 94%|█████████▎| 702/750 [36:24<02:20,  2.93s/it]
                                                 
{'loss': 0.1167, 'grad_norm': 0.29848286509513855, 'learning_rate': 9.050349822456143e-05, 'epoch': 4.68}
+
 94%|█████████▎| 702/750 [36:24<02:20,  2.93s/it]
 94%|█████████▎| 703/750 [36:26<02:12,  2.81s/it]
                                                 
{'loss': 0.1087, 'grad_norm': 0.3080123960971832, 'learning_rate': 9.043910583062168e-05, 'epoch': 4.69}
+
 94%|█████████▎| 703/750 [36:26<02:12,  2.81s/it]
 94%|█████████▍| 704/750 [36:30<02:25,  3.16s/it]
                                                 
{'loss': 0.1439, 'grad_norm': 0.35511383414268494, 'learning_rate': 9.037485068515265e-05, 'epoch': 4.69}
+
 94%|█████████▍| 704/750 [36:30<02:25,  3.16s/it]
 94%|█████████▍| 705/750 [36:32<02:00,  2.67s/it]
                                                 
{'loss': 0.1298, 'grad_norm': 0.3541339635848999, 'learning_rate': 9.03107323012848e-05, 'epoch': 4.7}
+
 94%|█████████▍| 705/750 [36:32<02:00,  2.67s/it]
 94%|█████████▍| 706/750 [36:37<02:36,  3.56s/it]
                                                 
{'loss': 0.1315, 'grad_norm': 0.30545008182525635, 'learning_rate': 9.024675019456317e-05, 'epoch': 4.71}
+
 94%|█████████▍| 706/750 [36:37<02:36,  3.56s/it]
 94%|█████████▍| 707/750 [36:41<02:29,  3.48s/it]
                                                 
{'loss': 0.1061, 'grad_norm': 0.3111902177333832, 'learning_rate': 9.01829038829319e-05, 'epoch': 4.71}
+
 94%|█████████▍| 707/750 [36:41<02:29,  3.48s/it]
 94%|█████████▍| 708/750 [36:43<02:06,  3.02s/it]
                                                 
{'loss': 0.1294, 'grad_norm': 0.32854029536247253, 'learning_rate': 9.011919288671903e-05, 'epoch': 4.72}
+
 94%|█████████▍| 708/750 [36:43<02:06,  3.02s/it]
 95%|█████████▍| 709/750 [36:45<01:58,  2.88s/it]
                                                 
{'loss': 0.1151, 'grad_norm': 0.27440062165260315, 'learning_rate': 9.005561672862141e-05, 'epoch': 4.73}
+
 95%|█████████▍| 709/750 [36:45<01:58,  2.88s/it]
 95%|█████████▍| 710/750 [36:49<02:02,  3.06s/it]
                                                 
{'loss': 0.1255, 'grad_norm': 0.2898067533969879, 'learning_rate': 8.999217493368951e-05, 'epoch': 4.73}
+
 95%|█████████▍| 710/750 [36:49<02:02,  3.06s/it]
 95%|█████████▍| 711/750 [36:52<02:05,  3.22s/it]
                                                 
{'loss': 0.1465, 'grad_norm': 0.29789605736732483, 'learning_rate': 8.992886702931274e-05, 'epoch': 4.74}
+
 95%|█████████▍| 711/750 [36:52<02:05,  3.22s/it]
 95%|█████████▍| 712/750 [36:55<01:58,  3.11s/it]
                                                 
{'loss': 0.1135, 'grad_norm': 0.30312469601631165, 'learning_rate': 8.986569254520445e-05, 'epoch': 4.75}
+
 95%|█████████▍| 712/750 [36:55<01:58,  3.11s/it]
 95%|█████████▌| 713/750 [36:59<01:59,  3.24s/it]
                                                 
{'loss': 0.1163, 'grad_norm': 0.2980082631111145, 'learning_rate': 8.980265101338746e-05, 'epoch': 4.75}
+
 95%|█████████▌| 713/750 [36:59<01:59,  3.24s/it]
 95%|█████████▌| 714/750 [37:01<01:48,  3.02s/it]
                                                 
{'loss': 0.0974, 'grad_norm': 0.28560250997543335, 'learning_rate': 8.973974196817932e-05, 'epoch': 4.76}
+
 95%|█████████▌| 714/750 [37:01<01:48,  3.02s/it]
 95%|█████████▌| 715/750 [37:04<01:38,  2.83s/it]
                                                 
{'loss': 0.1015, 'grad_norm': 0.29474154114723206, 'learning_rate': 8.967696494617803e-05, 'epoch': 4.77}
+
 95%|█████████▌| 715/750 [37:04<01:38,  2.83s/it]
 95%|█████████▌| 716/750 [37:06<01:32,  2.72s/it]
                                                 
{'loss': 0.1229, 'grad_norm': 0.3295375406742096, 'learning_rate': 8.961431948624752e-05, 'epoch': 4.77}
+
 95%|█████████▌| 716/750 [37:06<01:32,  2.72s/it]
 96%|█████████▌| 717/750 [37:09<01:27,  2.66s/it]
                                                 
{'loss': 0.1043, 'grad_norm': 0.26680976152420044, 'learning_rate': 8.955180512950356e-05, 'epoch': 4.78}
+
 96%|█████████▌| 717/750 [37:09<01:27,  2.66s/it]
 96%|█████████▌| 718/750 [37:11<01:20,  2.52s/it]
                                                 
{'loss': 0.0916, 'grad_norm': 0.2683340609073639, 'learning_rate': 8.948942141929963e-05, 'epoch': 4.79}
+
 96%|█████████▌| 718/750 [37:11<01:20,  2.52s/it]
 96%|█████████▌| 719/750 [37:13<01:19,  2.57s/it]
                                                 
{'loss': 0.1262, 'grad_norm': 0.34575700759887695, 'learning_rate': 8.942716790121283e-05, 'epoch': 4.79}
+
 96%|█████████▌| 719/750 [37:13<01:19,  2.57s/it]
 96%|█████████▌| 720/750 [37:16<01:20,  2.70s/it]
                                                 
{'loss': 0.1009, 'grad_norm': 0.2702127993106842, 'learning_rate': 8.936504412303008e-05, 'epoch': 4.8}
+
 96%|█████████▌| 720/750 [37:16<01:20,  2.70s/it]
 96%|█████████▌| 721/750 [37:20<01:21,  2.81s/it]
                                                 
{'loss': 0.1264, 'grad_norm': 0.297188937664032, 'learning_rate': 8.93030496347342e-05, 'epoch': 4.81}
+
 96%|█████████▌| 721/750 [37:20<01:21,  2.81s/it]
 96%|█████████▋| 722/750 [37:22<01:17,  2.78s/it]
                                                 
{'loss': 0.127, 'grad_norm': 0.32612112164497375, 'learning_rate': 8.924118398849037e-05, 'epoch': 4.81}
+
 96%|█████████▋| 722/750 [37:22<01:17,  2.78s/it]
 96%|█████████▋| 723/750 [37:25<01:14,  2.75s/it]
                                                 
{'loss': 0.157, 'grad_norm': 0.36822813749313354, 'learning_rate': 8.917944673863242e-05, 'epoch': 4.82}
+
 96%|█████████▋| 723/750 [37:25<01:14,  2.75s/it]
 97%|█████████▋| 724/750 [37:27<01:08,  2.62s/it]
                                                 
{'loss': 0.127, 'grad_norm': 0.3232595920562744, 'learning_rate': 8.911783744164937e-05, 'epoch': 4.83}
+
 97%|█████████▋| 724/750 [37:27<01:08,  2.62s/it]
 97%|█████████▋| 725/750 [37:30<01:08,  2.74s/it]
                                                 
{'loss': 0.1216, 'grad_norm': 0.3221473693847656, 'learning_rate': 8.905635565617214e-05, 'epoch': 4.83}
+
 97%|█████████▋| 725/750 [37:30<01:08,  2.74s/it]
 97%|█████████▋| 726/750 [37:35<01:18,  3.27s/it]
                                                 
{'loss': 0.128, 'grad_norm': 0.3113526701927185, 'learning_rate': 8.899500094296009e-05, 'epoch': 4.84}
+
 97%|█████████▋| 726/750 [37:35<01:18,  3.27s/it]
 97%|█████████▋| 727/750 [37:37<01:11,  3.10s/it]
                                                 
{'loss': 0.1101, 'grad_norm': 0.3306347727775574, 'learning_rate': 8.893377286488804e-05, 'epoch': 4.85}
+
 97%|█████████▋| 727/750 [37:37<01:11,  3.10s/it]
 97%|█████████▋| 728/750 [37:40<01:04,  2.92s/it]
                                                 
{'loss': 0.1084, 'grad_norm': 0.2810794413089752, 'learning_rate': 8.887267098693303e-05, 'epoch': 4.85}
+
 97%|█████████▋| 728/750 [37:40<01:04,  2.92s/it]
 97%|█████████▋| 729/750 [37:44<01:07,  3.20s/it]
                                                 
{'loss': 0.1231, 'grad_norm': 0.2879321873188019, 'learning_rate': 8.881169487616148e-05, 'epoch': 4.86}
+
 97%|█████████▋| 729/750 [37:44<01:07,  3.20s/it]
 97%|█████████▋| 730/750 [37:47<01:01,  3.07s/it]
                                                 
{'loss': 0.0974, 'grad_norm': 0.3002558946609497, 'learning_rate': 8.875084410171615e-05, 'epoch': 4.87}
+
 97%|█████████▋| 730/750 [37:47<01:01,  3.07s/it]
 97%|█████████▋| 731/750 [37:49<00:53,  2.80s/it]
                                                 
{'loss': 0.124, 'grad_norm': 0.2999451160430908, 'learning_rate': 8.869011823480348e-05, 'epoch': 4.87}
+
 97%|█████████▋| 731/750 [37:49<00:53,  2.80s/it]
 98%|█████████▊| 732/750 [37:52<00:50,  2.80s/it]
                                                 
{'loss': 0.1235, 'grad_norm': 0.33787062764167786, 'learning_rate': 8.862951684868085e-05, 'epoch': 4.88}
+
 98%|█████████▊| 732/750 [37:52<00:50,  2.80s/it]
 98%|█████████▊| 733/750 [37:55<00:49,  2.93s/it]
                                                 
{'loss': 0.1211, 'grad_norm': 0.341869056224823, 'learning_rate': 8.856903951864397e-05, 'epoch': 4.89}
+
 98%|█████████▊| 733/750 [37:55<00:49,  2.93s/it]
 98%|█████████▊| 734/750 [37:58<00:47,  2.95s/it]
                                                 
{'loss': 0.1066, 'grad_norm': 0.34291374683380127, 'learning_rate': 8.85086858220144e-05, 'epoch': 4.89}
+
 98%|█████████▊| 734/750 [37:58<00:47,  2.95s/it]
 98%|█████████▊| 735/750 [38:00<00:43,  2.87s/it]
                                                 
{'loss': 0.1413, 'grad_norm': 0.3777810037136078, 'learning_rate': 8.844845533812719e-05, 'epoch': 4.9}
+
 98%|█████████▊| 735/750 [38:00<00:43,  2.87s/it]
 98%|█████████▊| 736/750 [38:03<00:38,  2.73s/it]
                                                 
{'loss': 0.1058, 'grad_norm': 0.36918580532073975, 'learning_rate': 8.838834764831844e-05, 'epoch': 4.91}
+
 98%|█████████▊| 736/750 [38:03<00:38,  2.73s/it]
 98%|█████████▊| 737/750 [38:05<00:32,  2.53s/it]
                                                 
{'loss': 0.1257, 'grad_norm': 0.32615846395492554, 'learning_rate': 8.83283623359132e-05, 'epoch': 4.91}
+
 98%|█████████▊| 737/750 [38:05<00:32,  2.53s/it]
 98%|█████████▊| 738/750 [38:07<00:29,  2.48s/it]
                                                 
{'loss': 0.1082, 'grad_norm': 0.2998989224433899, 'learning_rate': 8.826849898621327e-05, 'epoch': 4.92}
+
 98%|█████████▊| 738/750 [38:07<00:29,  2.48s/it]
 99%|█████████▊| 739/750 [38:10<00:27,  2.54s/it]
                                                 
{'loss': 0.1097, 'grad_norm': 0.32782962918281555, 'learning_rate': 8.82087571864852e-05, 'epoch': 4.93}
+
 99%|█████████▊| 739/750 [38:10<00:27,  2.54s/it]
 99%|█████████▊| 740/750 [38:13<00:25,  2.58s/it]
                                                 
{'loss': 0.1199, 'grad_norm': 0.2914181053638458, 'learning_rate': 8.814913652594829e-05, 'epoch': 4.93}
+
 99%|█████████▊| 740/750 [38:13<00:25,  2.58s/it]
 99%|█████████▉| 741/750 [38:15<00:23,  2.61s/it]
                                                 
{'loss': 0.1332, 'grad_norm': 0.31852415204048157, 'learning_rate': 8.808963659576277e-05, 'epoch': 4.94}
+
 99%|█████████▉| 741/750 [38:15<00:23,  2.61s/it]
 99%|█████████▉| 742/750 [38:18<00:20,  2.58s/it]
                                                 
{'loss': 0.1262, 'grad_norm': 0.31790265440940857, 'learning_rate': 8.803025698901805e-05, 'epoch': 4.95}
+
 99%|█████████▉| 742/750 [38:18<00:20,  2.58s/it]
 99%|█████████▉| 743/750 [38:21<00:19,  2.77s/it]
                                                 
{'loss': 0.1038, 'grad_norm': 0.2953394949436188, 'learning_rate': 8.797099730072091e-05, 'epoch': 4.95}
+
 99%|█████████▉| 743/750 [38:21<00:19,  2.77s/it]
 99%|█████████▉| 744/750 [38:23<00:15,  2.60s/it]
                                                 
{'loss': 0.1, 'grad_norm': 0.27932432293891907, 'learning_rate': 8.791185712778405e-05, 'epoch': 4.96}
+
 99%|█████████▉| 744/750 [38:23<00:15,  2.60s/it]
 99%|█████████▉| 745/750 [38:26<00:13,  2.60s/it]
                                                 
{'loss': 0.1195, 'grad_norm': 0.3664175570011139, 'learning_rate': 8.785283606901446e-05, 'epoch': 4.97}
+
 99%|█████████▉| 745/750 [38:26<00:13,  2.60s/it]
 99%|█████████▉| 746/750 [38:28<00:09,  2.47s/it]
                                                 
{'loss': 0.162, 'grad_norm': 0.4087444841861725, 'learning_rate': 8.779393372510207e-05, 'epoch': 4.97}
+
 99%|█████████▉| 746/750 [38:28<00:09,  2.47s/it]
100%|█████████▉| 747/750 [38:31<00:07,  2.49s/it]
                                                 
{'loss': 0.1082, 'grad_norm': 0.3134411573410034, 'learning_rate': 8.773514969860834e-05, 'epoch': 4.98}
+
100%|█████████▉| 747/750 [38:31<00:07,  2.49s/it]
100%|█████████▉| 748/750 [38:33<00:05,  2.51s/it]
                                                 
{'loss': 0.126, 'grad_norm': 0.31796225905418396, 'learning_rate': 8.767648359395506e-05, 'epoch': 4.99}
+
100%|█████████▉| 748/750 [38:33<00:05,  2.51s/it]
100%|█████████▉| 749/750 [38:36<00:02,  2.63s/it]
                                                 
{'loss': 0.1055, 'grad_norm': 0.30049628019332886, 'learning_rate': 8.761793501741308e-05, 'epoch': 4.99}
+
100%|█████████▉| 749/750 [38:36<00:02,  2.63s/it]
100%|██████████| 750/750 [38:41<00:00,  3.20s/it]
                                                 
{'loss': 0.1274, 'grad_norm': 0.32126384973526, 'learning_rate': 8.755950357709131e-05, 'epoch': 5.0}
+
100%|██████████| 750/750 [38:41<00:00,  3.20s/it][INFO|trainer.py:3515] 2025-06-27 01:47:32,751 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-750
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685daadd-0d84c1ce7921b2302b4b9340;0c01b7c6-a810-419a-9c99-fae4a7d9888b)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-27 01:47:33,487 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-750/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-27 01:47:33,487 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-750/special_tokens_map.json
+[2025-06-27 01:47:34,516] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Checkpoint global_step750 is begin to save!
+[2025-06-27 01:47:34,539] [INFO] [logging.py:107:log_dist] [Rank 0] Saving model checkpoint: ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-750/global_step750/mp_rank_00_model_states.pt
+[INFO|trainer.py:3607] 2025-06-27 01:47:34,659 >> Deleting older checkpoint [outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-600] due to args.save_total_limit
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685daade-6176991d00c70d45281aff8e;77a9104c-e13c-4e66-9515-bce196157d6d)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685daade-33111cb847f52fd2740482ee;609176c6-5011-4764-a0b2-0d720d46c69f)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+[INFO|trainer.py:2406] 2025-06-27 01:47:35,029 >> 
+
+Training completed. Do not forget to share your model on huggingface.co/models =)
+
+
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685daadf-0f5624d61fce80f1581543ce;dc7f40bf-42e3-46e4-b8cb-668ddc829d27)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685daadf-180d73c50f2b099e0ebb4964;34b8c08a-3db8-4625-92d8-c7ca2d0811cd)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+[INFO|trainer.py:2644] 2025-06-27 01:47:35,394 >> Loading best model from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-200 (score: 0.2748406231403351).
+[INFO|deepspeed.py:431] 2025-06-27 01:47:35,395 >> Attempting to resume from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-200
+[2025-06-27 01:47:35,396] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Begin Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-200/global_step200/mp_rank_00_model_states.pt...
+[2025-06-27 01:47:35,415] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] End Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-200/global_step200/mp_rank_00_model_states.pt...
+[2025-06-27 01:47:35,416] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Begin Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-200/global_step200/mp_rank_00_model_states.pt...
+[2025-06-27 01:47:35,431] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] End Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-200/global_step200/mp_rank_00_model_states.pt...
+[2025-06-27 01:47:35,464] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Begin Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt...
+[2025-06-27 01:47:35,485] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] End Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-200/global_step200/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt...
+[2025-06-27 01:47:35,485] [INFO] [engine.py:3277:_get_all_zero_checkpoint_state_dicts] successfully read 4 ZeRO state_dicts for rank 0
+[2025-06-27 01:47:35,493] [INFO] [engine.py:3227:_load_zero_checkpoint] loading 4 zero partition checkpoints for rank 0
+
                                                 
{'train_runtime': 2339.7328, 'train_samples_per_second': 32.055, 'train_steps_per_second': 0.321, 'train_loss': 0.21062097707390784, 'epoch': 5.0}
+
100%|██████████| 750/750 [38:57<00:00,  3.20s/it][INFO|trainer.py:2447] 2025-06-27 01:47:35,692 >> Deleting older checkpoint [outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/checkpoint-750] due to args.save_total_limit
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685daadf-40cb2a8928dde2cb54345541;0dd51022-43b5-4aae-8034-5476fe7bc603)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685daadf-6c07a79643641998571f8cac;26c42325-fd56-44c0-af73-1ed2358f01c1)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685daadf-79a5aa2457f23e0b190127a1;7b42c3e9-af24-41cf-bfd9-562b6fbf2b4c)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+
100%|██████████| 750/750 [38:58<00:00,  3.12s/it]
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685daae0-0deedf202d072be627c3fc6a;5fb86a95-2b84-4a7a-8570-bd6a594cb6ac)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+[INFO|trainer.py:3515] 2025-06-27 01:47:49,703 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685daaed-3c23ffa0761622e90c8938ab;cbf32f08-de08-41c2-af3e-7e9c2d9e9c4c)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-27 01:47:49,986 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-27 01:47:49,987 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/special_tokens_map.json
+***** train metrics *****
+  epoch                    =          5.0
+  total_flos               = 2403339915GF
+  train_loss               =       0.2106
+  train_runtime            =   0:38:59.73
+  train_samples            =        15000
+  train_samples_per_second =       32.055
+  train_steps_per_second   =        0.321
+06/27/2025 01:47:51 - INFO - __main__ - *** Evaluate ***
+[INFO|trainer.py:3831] 2025-06-27 01:47:51,187 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-27 01:47:51,187 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-27 01:47:51,187 >>   Batch size = 25
+
  0%|          | 0/10 [00:00<?, ?it/s]
 20%|██        | 2/10 [00:00<00:03,  2.36it/s]
 30%|███       | 3/10 [00:01<00:04,  1.73it/s]
 40%|████      | 4/10 [00:02<00:03,  1.62it/s]
 50%|█████     | 5/10 [00:03<00:04,  1.20it/s]
 60%|██████    | 6/10 [00:04<00:03,  1.21it/s]
 70%|███████   | 7/10 [00:05<00:02,  1.29it/s]
 80%|████████  | 8/10 [00:05<00:01,  1.41it/s]
 90%|█████████ | 9/10 [00:06<00:00,  1.30it/s]
100%|██████████| 10/10 [00:09<00:00,  1.33s/it]
100%|██████████| 10/10 [00:13<00:00,  1.33s/it]
+***** eval metrics *****
+  epoch                   =        5.0
+  eval_accuracy           =     0.4615
+  eval_loss               =     0.2748
+  eval_runtime            = 0:00:21.71
+  eval_samples            =       1000
+  eval_samples_per_second =     46.061
+  eval_steps_per_second   =      0.461
+  perplexity              =     1.3163
+[INFO|modelcard.py:449] 2025-06-27 01:48:13,116 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.46148841826604897}]}
+wandb: 
+wandb: 🚀 View run ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/en/baseline/data_15000_1000/ at: https://wandb.ai/indic-encoder/midalign/runs/dtuhsbl4
+wandb: Find logs at: wandb/run-20250627_010836-dtuhsbl4/logs
+[rank0]:[W627 01:48:17.647819481 ProcessGroupNCCL.cpp:1479] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
diff --git a/en/baseline/data_15000_1000/train_results.json b/en/baseline/data_15000_1000/train_results.json
new file mode 100644
index 0000000000000000000000000000000000000000..2af164a2e1b5d9f7ca962ca86acc1470be03ddd8
--- /dev/null
+++ b/en/baseline/data_15000_1000/train_results.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6be0ca5e38c201c00f65ff1ed2917ee8c279e981fa597864b02e9a8f73c13286
+size 238
diff --git a/en/baseline/data_15000_1000/trainer_state.json b/en/baseline/data_15000_1000/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..82ecbc03bab3c3767dd8322c32ce55eafda772bc
--- /dev/null
+++ b/en/baseline/data_15000_1000/trainer_state.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f34e6652ddc8797a127bb2e9aa3c4a0afd717af28f8d44987f09dfe3f306cf44
+size 129215
diff --git a/en/baseline/data_15000_1000/training_args.bin b/en/baseline/data_15000_1000/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..faffedb31b34c9635c09771f47b340cecfa5f06a
--- /dev/null
+++ b/en/baseline/data_15000_1000/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:919f8d1e8774d9c124d170364fd95fc7a9b43c2f0cfecf93b3d0c961ad69acb1
+size 7761
diff --git a/te/baseline/data_15000_1000/README.md b/te/baseline/data_15000_1000/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..34b35b754db73a85ebfd9afc05ddebeda7909aa1
--- /dev/null
+++ b/te/baseline/data_15000_1000/README.md
@@ -0,0 +1,70 @@
+---
+license: llama3.1
+base_model: meta-llama/Llama-3.1-8B-Instruct
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+library_name: peft
+model-index:
+- name: data_15000_1000
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# data_15000_1000
+
+This model is a fine-tuned version of [meta-llama/Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.1428
+- Accuracy: 0.2946
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0005
+- train_batch_size: 25
+- eval_batch_size: 25
+- seed: 1
+- distributed_type: multi-GPU
+- num_devices: 4
+- total_train_batch_size: 100
+- total_eval_batch_size: 100
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: inverse_sqrt
+- lr_scheduler_warmup_ratio: 0.03
+- num_epochs: 5.0
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Accuracy |
+|:-------------:|:------:|:----:|:---------------:|:--------:|
+| No log        | 0      | 0    | 0.2284          | 0.2880   |
+| 0.1383        | 1.3333 | 200  | 0.1454          | 0.2941   |
+| 0.1049        | 2.6667 | 400  | 0.1428          | 0.2946   |
+| 0.0852        | 4.0    | 600  | 0.1470          | 0.2946   |
+
+
+### Framework versions
+
+- PEFT 0.15.2
+- Transformers 4.44.0.dev0
+- Pytorch 2.7.1+cu126
+- Datasets 3.6.0
+- Tokenizers 0.19.1
\ No newline at end of file
diff --git a/te/baseline/data_15000_1000/adapter_config.json b/te/baseline/data_15000_1000/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..99d05b72118c8b38e231b264fba3aaf407f3019f
--- /dev/null
+++ b/te/baseline/data_15000_1000/adapter_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d26e346680bcbce5d2979a3f022de6768843ee3a8fa07b631fad9652faaaca0
+size 863
diff --git a/te/baseline/data_15000_1000/adapter_model.safetensors b/te/baseline/data_15000_1000/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c1d3a77c5433f7aa3c9183da656988b954ddd2c8
--- /dev/null
+++ b/te/baseline/data_15000_1000/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b135ec59d7da0ca6ad95a7f7bddf5666e03109f7e074a1dc8be8d9aef7a63470
+size 42002584
diff --git a/te/baseline/data_15000_1000/adapter_model/README.md b/te/baseline/data_15000_1000/adapter_model/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0d31128190920e45b61115944d16e773c2ec94c3
--- /dev/null
+++ b/te/baseline/data_15000_1000/adapter_model/README.md
@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/te/baseline/data_15000_1000/adapter_model/adapter_config.json b/te/baseline/data_15000_1000/adapter_model/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..99d05b72118c8b38e231b264fba3aaf407f3019f
--- /dev/null
+++ b/te/baseline/data_15000_1000/adapter_model/adapter_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d26e346680bcbce5d2979a3f022de6768843ee3a8fa07b631fad9652faaaca0
+size 863
diff --git a/te/baseline/data_15000_1000/adapter_model/adapter_model.safetensors b/te/baseline/data_15000_1000/adapter_model/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c1d3a77c5433f7aa3c9183da656988b954ddd2c8
--- /dev/null
+++ b/te/baseline/data_15000_1000/adapter_model/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b135ec59d7da0ca6ad95a7f7bddf5666e03109f7e074a1dc8be8d9aef7a63470
+size 42002584
diff --git a/te/baseline/data_15000_1000/all_results.json b/te/baseline/data_15000_1000/all_results.json
new file mode 100644
index 0000000000000000000000000000000000000000..68758268c8b06de330076816bfc2a0cdaeb6bce5
--- /dev/null
+++ b/te/baseline/data_15000_1000/all_results.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97dcf9bcd8b8d75e6ad6dfb546ba28e82eef1770e5cb82caf2ab0f98f8e7c59e
+size 483
diff --git a/te/baseline/data_15000_1000/checkpoint-400/README.md b/te/baseline/data_15000_1000/checkpoint-400/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0d31128190920e45b61115944d16e773c2ec94c3
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/README.md
@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/te/baseline/data_15000_1000/checkpoint-400/adapter_config.json b/te/baseline/data_15000_1000/checkpoint-400/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..99d05b72118c8b38e231b264fba3aaf407f3019f
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/adapter_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d26e346680bcbce5d2979a3f022de6768843ee3a8fa07b631fad9652faaaca0
+size 863
diff --git a/te/baseline/data_15000_1000/checkpoint-400/adapter_model.safetensors b/te/baseline/data_15000_1000/checkpoint-400/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c1d3a77c5433f7aa3c9183da656988b954ddd2c8
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b135ec59d7da0ca6ad95a7f7bddf5666e03109f7e074a1dc8be8d9aef7a63470
+size 42002584
diff --git a/te/baseline/data_15000_1000/checkpoint-400/adapter_model/README.md b/te/baseline/data_15000_1000/checkpoint-400/adapter_model/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0d31128190920e45b61115944d16e773c2ec94c3
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/adapter_model/README.md
@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/te/baseline/data_15000_1000/checkpoint-400/adapter_model/adapter_config.json b/te/baseline/data_15000_1000/checkpoint-400/adapter_model/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..c5e01d1d8cf346eface92093f85a95bf78b233b0
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/adapter_model/adapter_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:651a2455318a4b86ff036974ac8a5c006d00cfdd3a6245c4db9f138229d7a692
+size 863
diff --git a/te/baseline/data_15000_1000/checkpoint-400/adapter_model/adapter_model.safetensors b/te/baseline/data_15000_1000/checkpoint-400/adapter_model/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c1d3a77c5433f7aa3c9183da656988b954ddd2c8
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/adapter_model/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b135ec59d7da0ca6ad95a7f7bddf5666e03109f7e074a1dc8be8d9aef7a63470
+size 42002584
diff --git a/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt b/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..efdc8b9707562151dc27a064573f2295a2d80a04
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b1b71396a6c399bf2a5cb980051ea8eaeaf5b1e77f9d6c17924789780268b16
+size 62930469
diff --git a/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt b/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..2c76e988716587c0e2dc050433dd792de38c1701
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f51002d37555cbc3daa6817fc4e6f59006170653b9373a15b55ba3f21154c838
+size 62930533
diff --git a/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt b/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..b70dda0bfc0b121df4643645bd219848699d8ced
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3343cd58ae8d58b872745508db6cd80e476023471fa1a5bb55d50b7bac6bb314
+size 62930597
diff --git a/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt b/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..c4246825bda14d0e34f061d48416b31145e4db16
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f700b4bfacad581a0a415957c6664132e18c9b8427a323c4846e772809153840
+size 62930597
diff --git a/te/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt b/te/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt
new file mode 100644
index 0000000000000000000000000000000000000000..2f86ec69b0b01a71494dd72d91e5417703a9058f
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75be16bf24efa158fa9930e170fd0d2e3fb4d8328133f0be3959b13bb0755ee1
+size 42291289
diff --git a/te/baseline/data_15000_1000/checkpoint-400/latest b/te/baseline/data_15000_1000/checkpoint-400/latest
new file mode 100644
index 0000000000000000000000000000000000000000..e5bdf58d4f29d34e909da25905fad376f73e7c29
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/latest
@@ -0,0 +1 @@
+global_step400
\ No newline at end of file
diff --git a/te/baseline/data_15000_1000/checkpoint-400/rng_state_0.pth b/te/baseline/data_15000_1000/checkpoint-400/rng_state_0.pth
new file mode 100644
index 0000000000000000000000000000000000000000..12f64195b70ce260364bfacbf829fda642cfef6c
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/rng_state_0.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8661da0e67d77bd3925680a56f88595ce28bf4f139e22336ad725b2eeb8ccdf9
+size 15429
diff --git a/te/baseline/data_15000_1000/checkpoint-400/rng_state_1.pth b/te/baseline/data_15000_1000/checkpoint-400/rng_state_1.pth
new file mode 100644
index 0000000000000000000000000000000000000000..3fba526f53e067c67bf285ae48c01aceb1fdcdd0
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/rng_state_1.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f749eb919614a58ebd01326bc9f71726b4ceeb7e5840c36ab3da1548aa7bf3de
+size 15429
diff --git a/te/baseline/data_15000_1000/checkpoint-400/rng_state_2.pth b/te/baseline/data_15000_1000/checkpoint-400/rng_state_2.pth
new file mode 100644
index 0000000000000000000000000000000000000000..c2ccbcf14d00fda0262d8cbbc96e8948f550950a
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/rng_state_2.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81958d77d6b111836f06960a69f570587ee93c4380747eebc36248f39969c6be
+size 15429
diff --git a/te/baseline/data_15000_1000/checkpoint-400/rng_state_3.pth b/te/baseline/data_15000_1000/checkpoint-400/rng_state_3.pth
new file mode 100644
index 0000000000000000000000000000000000000000..b55a9eb1b2d82a76894e4455d5ec78540b304fe7
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/rng_state_3.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf8b2ecaa4c5ba49b0d19a0e05ea103e641b5f8924697d47fc2ca06f162ca790
+size 15429
diff --git a/te/baseline/data_15000_1000/checkpoint-400/special_tokens_map.json b/te/baseline/data_15000_1000/checkpoint-400/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..4ed2bd8240878a7a0d4fd2c60cdc89f6d7a5f1e1
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/special_tokens_map.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83da1082df286d75a2984dc06ec439f4febc3d862ac55d199402e5d345f5773a
+size 372
diff --git a/te/baseline/data_15000_1000/checkpoint-400/tokenizer.json b/te/baseline/data_15000_1000/checkpoint-400/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..66cd9d7e0daec95eb10d16a63c615637dbbb7304
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79e3e522635f3171300913bb421464a87de6222182a0570b9b2ccba2a964b2b4
+size 9085657
diff --git a/te/baseline/data_15000_1000/checkpoint-400/tokenizer_config.json b/te/baseline/data_15000_1000/checkpoint-400/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..7144ada11807e90b92529f17434f8d01915c3dff
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/tokenizer_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d967a51bb800d3e471ea23dd3f7b054b136600238bbbb40612b8b96b0370746e
+size 55427
diff --git a/te/baseline/data_15000_1000/checkpoint-400/trainer_state.json b/te/baseline/data_15000_1000/checkpoint-400/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..73b4725dd91b4f0711a96c4b1a259c7ccd8a3a24
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/trainer_state.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ccda5dc2c26eb60231372f11bc3edc286b5a494f8f3e387c750087e8cd6d0b04
+size 69738
diff --git a/te/baseline/data_15000_1000/checkpoint-400/training_args.bin b/te/baseline/data_15000_1000/checkpoint-400/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..48852a5b49fabd0bc05f759b98bc85ab6d2de0cc
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb70f1f7eb340def6adc18e2dbd35ca24fe7cef2fe7c9151a297ff1bb9ed2a09
+size 7761
diff --git a/te/baseline/data_15000_1000/checkpoint-400/zero_to_fp32.py b/te/baseline/data_15000_1000/checkpoint-400/zero_to_fp32.py
new file mode 100755
index 0000000000000000000000000000000000000000..0e759146cadd92ddfefab3680146c2bd6a2b5c04
--- /dev/null
+++ b/te/baseline/data_15000_1000/checkpoint-400/zero_to_fp32.py
@@ -0,0 +1,760 @@
+#!/usr/bin/env python
+
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: Apache-2.0
+
+# DeepSpeed Team
+
+# This script extracts fp32 consolidated weights from a zero 1, 2 and 3 DeepSpeed checkpoints. It gets
+# copied into the top level checkpoint dir, so the user can easily do the conversion at any point in
+# the future. Once extracted, the weights don't require DeepSpeed and can be used in any
+# application.
+#
+# example:
+#   python zero_to_fp32.py . output_dir/
+#   or
+#   python zero_to_fp32.py . output_dir/ --safe_serialization
+
+import argparse
+import torch
+import glob
+import math
+import os
+import re
+import gc
+import json
+import numpy as np
+from tqdm import tqdm
+from collections import OrderedDict
+from dataclasses import dataclass
+
+# while this script doesn't use deepspeed to recover data, since the checkpoints are pickled with
+# DeepSpeed data structures it has to be available in the current python environment.
+from deepspeed.utils import logger
+from deepspeed.checkpoint.constants import (DS_VERSION, OPTIMIZER_STATE_DICT, SINGLE_PARTITION_OF_FP32_GROUPS,
+                                            FP32_FLAT_GROUPS, ZERO_STAGE, PARTITION_COUNT, PARAM_SHAPES, BUFFER_NAMES,
+                                            FROZEN_PARAM_SHAPES, FROZEN_PARAM_FRAGMENTS)
+
+
+@dataclass
+class zero_model_state:
+    buffers: dict()
+    param_shapes: dict()
+    shared_params: list
+    ds_version: int
+    frozen_param_shapes: dict()
+    frozen_param_fragments: dict()
+
+
+debug = 0
+
+# load to cpu
+device = torch.device('cpu')
+
+
+def atoi(text):
+    return int(text) if text.isdigit() else text
+
+
+def natural_keys(text):
+    '''
+    alist.sort(key=natural_keys) sorts in human order
+    http://nedbatchelder.com/blog/200712/human_sorting.html
+    (See Toothy's implementation in the comments)
+    '''
+    return [atoi(c) for c in re.split(r'(\d+)', text)]
+
+
+def get_model_state_file(checkpoint_dir, zero_stage):
+    if not os.path.isdir(checkpoint_dir):
+        raise FileNotFoundError(f"Directory '{checkpoint_dir}' doesn't exist")
+
+    # there should be only one file
+    if zero_stage <= 2:
+        file = os.path.join(checkpoint_dir, "mp_rank_00_model_states.pt")
+    elif zero_stage == 3:
+        file = os.path.join(checkpoint_dir, "zero_pp_rank_0_mp_rank_00_model_states.pt")
+
+    if not os.path.exists(file):
+        raise FileNotFoundError(f"can't find model states file at '{file}'")
+
+    return file
+
+
+def get_checkpoint_files(checkpoint_dir, glob_pattern):
+    # XXX: need to test that this simple glob rule works for multi-node setup too
+    ckpt_files = sorted(glob.glob(os.path.join(checkpoint_dir, glob_pattern)), key=natural_keys)
+
+    if len(ckpt_files) == 0:
+        raise FileNotFoundError(f"can't find {glob_pattern} files in directory '{checkpoint_dir}'")
+
+    return ckpt_files
+
+
+def get_optim_files(checkpoint_dir):
+    return get_checkpoint_files(checkpoint_dir, "*_optim_states.pt")
+
+
+def get_model_state_files(checkpoint_dir):
+    return get_checkpoint_files(checkpoint_dir, "*_model_states.pt")
+
+
+def parse_model_states(files):
+    zero_model_states = []
+    for file in files:
+        state_dict = torch.load(file, map_location=device, weights_only=False)
+
+        if BUFFER_NAMES not in state_dict:
+            raise ValueError(f"{file} is not a model state checkpoint")
+        buffer_names = state_dict[BUFFER_NAMES]
+        if debug:
+            print("Found buffers:", buffer_names)
+
+        # recover just the buffers while restoring them to fp32 if they were saved in fp16
+        buffers = {k: v.float() for k, v in state_dict["module"].items() if k in buffer_names}
+        param_shapes = state_dict[PARAM_SHAPES]
+
+        # collect parameters that are included in param_shapes
+        param_names = []
+        for s in param_shapes:
+            for name in s.keys():
+                param_names.append(name)
+
+        # update with frozen parameters
+        frozen_param_shapes = state_dict.get(FROZEN_PARAM_SHAPES, None)
+        if frozen_param_shapes is not None:
+            if debug:
+                print(f"Found frozen_param_shapes: {frozen_param_shapes}")
+            param_names += list(frozen_param_shapes.keys())
+
+        # handle shared params
+        shared_params = [[k, v] for k, v in state_dict["shared_params"].items()]
+
+        ds_version = state_dict.get(DS_VERSION, None)
+
+        frozen_param_fragments = state_dict.get(FROZEN_PARAM_FRAGMENTS, None)
+
+        z_model_state = zero_model_state(buffers=buffers,
+                                         param_shapes=param_shapes,
+                                         shared_params=shared_params,
+                                         ds_version=ds_version,
+                                         frozen_param_shapes=frozen_param_shapes,
+                                         frozen_param_fragments=frozen_param_fragments)
+        zero_model_states.append(z_model_state)
+
+    return zero_model_states
+
+
+def parse_optim_states(files, ds_checkpoint_dir):
+    total_files = len(files)
+    state_dicts = []
+    for f in tqdm(files, desc='Loading checkpoint shards'):
+        state_dict = torch.load(f, map_location=device, mmap=True, weights_only=False)
+        # immediately discard the potentially huge 2 optimizer states as we only care for fp32 master weights
+        # and also handle the case where it was already removed by another helper script
+        state_dict["optimizer_state_dict"].pop("optimizer_state_dict", None)
+        state_dicts.append(state_dict)
+
+    if not ZERO_STAGE in state_dicts[0][OPTIMIZER_STATE_DICT]:
+        raise ValueError(f"{files[0]} is not a zero checkpoint")
+    zero_stage = state_dicts[0][OPTIMIZER_STATE_DICT][ZERO_STAGE]
+    world_size = state_dicts[0][OPTIMIZER_STATE_DICT][PARTITION_COUNT]
+
+    # For ZeRO-2 each param group can have different partition_count as data parallelism for expert
+    # parameters can be different from data parallelism for non-expert parameters. So we can just
+    # use the max of the partition_count to get the dp world_size.
+
+    if type(world_size) is list:
+        world_size = max(world_size)
+
+    if world_size != total_files:
+        raise ValueError(
+            f"Expected {world_size} of '*_optim_states.pt' under '{ds_checkpoint_dir}' but found {total_files} files. "
+            "Possibly due to an overwrite of an old checkpoint, or a checkpoint didn't get saved by one or more processes."
+        )
+
+    # the groups are named differently in each stage
+    if zero_stage <= 2:
+        fp32_groups_key = SINGLE_PARTITION_OF_FP32_GROUPS
+    elif zero_stage == 3:
+        fp32_groups_key = FP32_FLAT_GROUPS
+    else:
+        raise ValueError(f"unknown zero stage {zero_stage}")
+
+    fp32_flat_groups = [state_dicts[i][OPTIMIZER_STATE_DICT][fp32_groups_key] for i in range(len(state_dicts))]
+    return zero_stage, world_size, fp32_flat_groups
+
+
+def _get_fp32_state_dict_from_zero_checkpoint(ds_checkpoint_dir, exclude_frozen_parameters):
+    """
+    Returns fp32 state_dict reconstructed from ds checkpoint
+
+    Args:
+        - ``ds_checkpoint_dir``: path to the deepspeed checkpoint folder (where the optimizer files are)
+
+    """
+    print(f"Processing zero checkpoint '{ds_checkpoint_dir}'")
+
+    optim_files = get_optim_files(ds_checkpoint_dir)
+    zero_stage, world_size, fp32_flat_groups = parse_optim_states(optim_files, ds_checkpoint_dir)
+    print(f"Detected checkpoint of type zero stage {zero_stage}, world_size: {world_size}")
+
+    model_files = get_model_state_files(ds_checkpoint_dir)
+
+    zero_model_states = parse_model_states(model_files)
+    print(f'Parsing checkpoint created by deepspeed=={zero_model_states[0].ds_version}')
+
+    if zero_stage <= 2:
+        return _get_fp32_state_dict_from_zero2_checkpoint(world_size, fp32_flat_groups, zero_model_states,
+                                                          exclude_frozen_parameters)
+    elif zero_stage == 3:
+        return _get_fp32_state_dict_from_zero3_checkpoint(world_size, fp32_flat_groups, zero_model_states,
+                                                          exclude_frozen_parameters)
+
+
+def _zero2_merge_frozen_params(state_dict, zero_model_states):
+    if zero_model_states[0].frozen_param_shapes is None or len(zero_model_states[0].frozen_param_shapes) == 0:
+        return
+
+    frozen_param_shapes = zero_model_states[0].frozen_param_shapes
+    frozen_param_fragments = zero_model_states[0].frozen_param_fragments
+
+    if debug:
+        num_elem = sum(s.numel() for s in frozen_param_shapes.values())
+        print(f'rank 0: {FROZEN_PARAM_SHAPES}.numel = {num_elem}')
+
+        wanted_params = len(frozen_param_shapes)
+        wanted_numel = sum(s.numel() for s in frozen_param_shapes.values())
+        avail_numel = sum([p.numel() for p in frozen_param_fragments.values()])
+        print(f'Frozen params: Have {avail_numel} numels to process.')
+        print(f'Frozen params: Need {wanted_numel} numels in {wanted_params} params')
+
+    total_params = 0
+    total_numel = 0
+    for name, shape in frozen_param_shapes.items():
+        total_params += 1
+        unpartitioned_numel = shape.numel()
+        total_numel += unpartitioned_numel
+
+        state_dict[name] = frozen_param_fragments[name]
+
+        if debug:
+            print(f"{name} full shape: {shape} unpartitioned numel {unpartitioned_numel} ")
+
+    print(f"Reconstructed Frozen fp32 state dict with {total_params} params {total_numel} elements")
+
+
+def _has_callable(obj, fn):
+    attr = getattr(obj, fn, None)
+    return callable(attr)
+
+
+def _zero2_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states):
+    param_shapes = zero_model_states[0].param_shapes
+
+    # Reconstruction protocol:
+    #
+    # XXX: document this
+
+    if debug:
+        for i in range(world_size):
+            for j in range(len(fp32_flat_groups[0])):
+                print(f"{FP32_FLAT_GROUPS}[{i}][{j}].shape={fp32_flat_groups[i][j].shape}")
+
+    # XXX: memory usage doubles here (zero2)
+    num_param_groups = len(fp32_flat_groups[0])
+    merged_single_partition_of_fp32_groups = []
+    for i in range(num_param_groups):
+        merged_partitions = [sd[i] for sd in fp32_flat_groups]
+        full_single_fp32_vector = torch.cat(merged_partitions, 0)
+        merged_single_partition_of_fp32_groups.append(full_single_fp32_vector)
+    avail_numel = sum(
+        [full_single_fp32_vector.numel() for full_single_fp32_vector in merged_single_partition_of_fp32_groups])
+
+    if debug:
+        wanted_params = sum([len(shapes) for shapes in param_shapes])
+        wanted_numel = sum([sum(shape.numel() for shape in shapes.values()) for shapes in param_shapes])
+        # not asserting if there is a mismatch due to possible padding
+        print(f"Have {avail_numel} numels to process.")
+        print(f"Need {wanted_numel} numels in {wanted_params} params.")
+
+    # params
+    # XXX: for huge models that can't fit into the host's RAM we will have to recode this to support
+    # out-of-core computing solution
+    total_numel = 0
+    total_params = 0
+    for shapes, full_single_fp32_vector in zip(param_shapes, merged_single_partition_of_fp32_groups):
+        offset = 0
+        avail_numel = full_single_fp32_vector.numel()
+        for name, shape in shapes.items():
+
+            unpartitioned_numel = shape.numel() if _has_callable(shape, 'numel') else math.prod(shape)
+            total_numel += unpartitioned_numel
+            total_params += 1
+
+            if debug:
+                print(f"{name} full shape: {shape} unpartitioned numel {unpartitioned_numel} ")
+            state_dict[name] = full_single_fp32_vector.narrow(0, offset, unpartitioned_numel).view(shape)
+            offset += unpartitioned_numel
+
+        # Z2 started to align to 2*world_size to improve nccl performance. Therefore both offset and
+        # avail_numel can differ by anywhere between 0..2*world_size. Due to two unrelated complex
+        # paddings performed in the code it's almost impossible to predict the exact numbers w/o the
+        # live optimizer object, so we are checking that the numbers are within the right range
+        align_to = 2 * world_size
+
+        def zero2_align(x):
+            return align_to * math.ceil(x / align_to)
+
+        if debug:
+            print(f"original offset={offset}, avail_numel={avail_numel}")
+
+        offset = zero2_align(offset)
+        avail_numel = zero2_align(avail_numel)
+
+        if debug:
+            print(f"aligned  offset={offset}, avail_numel={avail_numel}")
+
+        # Sanity check
+        if offset != avail_numel:
+            raise ValueError(f"consumed {offset} numels out of {avail_numel} - something is wrong")
+
+    print(f"Reconstructed fp32 state dict with {total_params} params {total_numel} elements")
+
+
+def _get_fp32_state_dict_from_zero2_checkpoint(world_size, fp32_flat_groups, zero_model_states,
+                                               exclude_frozen_parameters):
+    state_dict = OrderedDict()
+
+    # buffers
+    buffers = zero_model_states[0].buffers
+    state_dict.update(buffers)
+    if debug:
+        print(f"added {len(buffers)} buffers")
+
+    if not exclude_frozen_parameters:
+        _zero2_merge_frozen_params(state_dict, zero_model_states)
+
+    _zero2_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states)
+
+    # recover shared parameters
+    for pair in zero_model_states[0].shared_params:
+        if pair[1] in state_dict:
+            state_dict[pair[0]] = state_dict[pair[1]]
+
+    return state_dict
+
+
+def zero3_partitioned_param_info(unpartitioned_numel, world_size):
+    remainder = unpartitioned_numel % world_size
+    padding_numel = (world_size - remainder) if remainder else 0
+    partitioned_numel = math.ceil(unpartitioned_numel / world_size)
+    return partitioned_numel, padding_numel
+
+
+def _zero3_merge_frozen_params(state_dict, world_size, zero_model_states):
+    if zero_model_states[0].frozen_param_shapes is None or len(zero_model_states[0].frozen_param_shapes) == 0:
+        return
+
+    if debug:
+        for i in range(world_size):
+            num_elem = sum(s.numel() for s in zero_model_states[i].frozen_param_fragments.values())
+            print(f'rank {i}: {FROZEN_PARAM_SHAPES}.numel = {num_elem}')
+
+        frozen_param_shapes = zero_model_states[0].frozen_param_shapes
+        wanted_params = len(frozen_param_shapes)
+        wanted_numel = sum(s.numel() for s in frozen_param_shapes.values())
+        avail_numel = sum([p.numel() for p in zero_model_states[0].frozen_param_fragments.values()]) * world_size
+        print(f'Frozen params: Have {avail_numel} numels to process.')
+        print(f'Frozen params: Need {wanted_numel} numels in {wanted_params} params')
+
+    total_params = 0
+    total_numel = 0
+    for name, shape in zero_model_states[0].frozen_param_shapes.items():
+        total_params += 1
+        unpartitioned_numel = shape.numel()
+        total_numel += unpartitioned_numel
+
+        param_frags = tuple(model_state.frozen_param_fragments[name] for model_state in zero_model_states)
+        state_dict[name] = torch.cat(param_frags, 0).narrow(0, 0, unpartitioned_numel).view(shape)
+
+        partitioned_numel, partitioned_padding_numel = zero3_partitioned_param_info(unpartitioned_numel, world_size)
+
+        if debug:
+            print(
+                f"Frozen params: {total_params} {name} full shape: {shape} partition0 numel={partitioned_numel} partitioned_padding_numel={partitioned_padding_numel}"
+            )
+
+    print(f"Reconstructed Frozen fp32 state dict with {total_params} params {total_numel} elements")
+
+
+class GatheredTensor:
+    """
+    A pseudo tensor that collects partitioned weights.
+    It is more memory efficient when there are multiple groups.
+    """
+
+    def __init__(self, flat_groups, flat_groups_offset, offset, partitioned_numel, shape):
+        self.flat_groups = flat_groups
+        self.flat_groups_offset = flat_groups_offset
+        self.offset = offset
+        self.partitioned_numel = partitioned_numel
+        self.shape = shape
+        self.dtype = self.flat_groups[0][0].dtype
+
+    def contiguous(self):
+        """
+        Merge partitioned weights from flat_groups into a single tensor.
+        """
+        end_idx = self.offset + self.partitioned_numel
+        world_size = len(self.flat_groups)
+        pad_flat_param_chunks = []
+
+        for rank_i in range(world_size):
+            # for each rank, we need to collect weights from related group/groups
+            flat_groups_at_rank_i = self.flat_groups[rank_i]
+            start_group_id = None
+            end_group_id = None
+            for group_id in range(len(self.flat_groups_offset)):
+                if self.flat_groups_offset[group_id] <= self.offset < self.flat_groups_offset[group_id + 1]:
+                    start_group_id = group_id
+                if self.flat_groups_offset[group_id] < end_idx <= self.flat_groups_offset[group_id + 1]:
+                    end_group_id = group_id
+                    break
+            # collect weights from related group/groups
+            for group_id in range(start_group_id, end_group_id + 1):
+                flat_tensor = flat_groups_at_rank_i[group_id]
+                start_offset = self.offset - self.flat_groups_offset[group_id]
+                end_offset = min(end_idx, self.flat_groups_offset[group_id + 1]) - self.flat_groups_offset[group_id]
+                pad_flat_param_chunks.append(flat_tensor[start_offset:end_offset])
+
+        # collect weights from all ranks
+        pad_flat_param = torch.cat(pad_flat_param_chunks, dim=0)
+        param = pad_flat_param[:self.shape.numel()].view(self.shape).contiguous()
+        return param
+
+
+def _zero3_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states):
+    param_shapes = zero_model_states[0].param_shapes
+    avail_numel = sum([flat_group.numel() for flat_group in fp32_flat_groups[0]]) * world_size
+
+    # Reconstruction protocol: For zero3 we need to zip the partitions together at boundary of each
+    # param, re-consolidating each param, while dealing with padding if any
+
+    # merge list of dicts, preserving order
+    param_shapes = {k: v for d in param_shapes for k, v in d.items()}
+
+    if debug:
+        for i in range(world_size):
+            print(f"{FP32_FLAT_GROUPS}[{i}].shape={fp32_flat_groups[i].shape}")
+
+        wanted_params = len(param_shapes)
+        wanted_numel = sum(shape.numel() for shape in param_shapes.values())
+        # not asserting if there is a mismatch due to possible padding
+        avail_numel = fp32_flat_groups[0].numel() * world_size
+        print(f"Trainable params: Have {avail_numel} numels to process.")
+        print(f"Trainable params: Need {wanted_numel} numels in {wanted_params} params.")
+
+    # params
+    # XXX: for huge models that can't fit into the host's RAM we will have to recode this to support
+    # out-of-core computing solution
+    offset = 0
+    total_numel = 0
+    total_params = 0
+    flat_groups_offset = [0] + list(np.cumsum([flat_tensor.numel() for flat_tensor in fp32_flat_groups[0]]))
+    for name, shape in tqdm(param_shapes.items(), desc='Gathering sharded weights'):
+        unpartitioned_numel = shape.numel()
+        total_numel += unpartitioned_numel
+        total_params += 1
+        partitioned_numel, partitioned_padding_numel = zero3_partitioned_param_info(unpartitioned_numel, world_size)
+
+        if debug:
+            print(
+                f"Trainable params: {total_params} {name} full shape: {shape} partition0 numel={partitioned_numel} partitioned_padding_numel={partitioned_padding_numel}"
+            )
+
+        # memory efficient tensor
+        tensor = GatheredTensor(fp32_flat_groups, flat_groups_offset, offset, partitioned_numel, shape)
+        state_dict[name] = tensor
+        offset += partitioned_numel
+
+    offset *= world_size
+
+    # Sanity check
+    if offset != avail_numel:
+        raise ValueError(f"consumed {offset} numels out of {avail_numel} - something is wrong")
+
+    print(f"Reconstructed Trainable fp32 state dict with {total_params} params {total_numel} elements")
+
+
+def _get_fp32_state_dict_from_zero3_checkpoint(world_size, fp32_flat_groups, zero_model_states,
+                                               exclude_frozen_parameters):
+    state_dict = OrderedDict()
+
+    # buffers
+    buffers = zero_model_states[0].buffers
+    state_dict.update(buffers)
+    if debug:
+        print(f"added {len(buffers)} buffers")
+
+    if not exclude_frozen_parameters:
+        _zero3_merge_frozen_params(state_dict, world_size, zero_model_states)
+
+    _zero3_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states)
+
+    # recover shared parameters
+    for pair in zero_model_states[0].shared_params:
+        if pair[1] in state_dict:
+            state_dict[pair[0]] = state_dict[pair[1]]
+
+    return state_dict
+
+
+def to_torch_tensor(state_dict, return_empty_tensor=False):
+    """
+    Convert state_dict of GatheredTensor to torch tensor
+    """
+    torch_state_dict = {}
+    converted_tensors = {}
+    for name, tensor in state_dict.items():
+        tensor_id = id(tensor)
+        if tensor_id in converted_tensors:  # shared tensors
+            shared_tensor = torch_state_dict[converted_tensors[tensor_id]]
+            torch_state_dict[name] = shared_tensor
+        else:
+            converted_tensors[tensor_id] = name
+            if return_empty_tensor:
+                torch_state_dict[name] = torch.empty(tensor.shape, dtype=tensor.dtype)
+            else:
+                torch_state_dict[name] = tensor.contiguous()
+    return torch_state_dict
+
+
+def get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir,
+                                             tag=None,
+                                             exclude_frozen_parameters=False,
+                                             lazy_mode=False):
+    """
+    Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated state_dict that can be loaded with
+    ``load_state_dict()`` and used for training without DeepSpeed or shared with others, for example
+    via a model hub.
+
+    Args:
+        - ``checkpoint_dir``: path to the desired checkpoint folder
+        - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in 'latest' file. e.g., ``global_step14``
+        - ``exclude_frozen_parameters``: exclude frozen parameters
+        - ``lazy_mode``: get state_dict in lazy mode. It returns a dict of pesduo tensor instead of torch tensor, which is more memory efficient.
+          Convert the pesduo tensor to torch tensor by ``.contiguous()``
+
+    Returns:
+        - pytorch ``state_dict``
+
+    A typical usage might be ::
+
+        from deepspeed.utils.zero_to_fp32 import get_fp32_state_dict_from_zero_checkpoint
+        # do the training and checkpoint saving
+        state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir) # already on cpu
+        model = model.cpu() # move to cpu
+        model.load_state_dict(state_dict)
+        # submit to model hub or save the model to share with others
+
+    In this example the ``model`` will no longer be usable in the deepspeed context of the same
+    application. i.e. you will need to re-initialize the deepspeed engine, since
+    ``model.load_state_dict(state_dict)`` will remove all the deepspeed magic from it.
+
+    If you want it all done for you, use ``load_state_dict_from_zero_checkpoint`` instead.
+
+    Note: the above usage may not work if your application doesn't have sufficient free CPU memory.
+    You may need to use the offline approach using the ``zero_to_fp32.py`` script that is saved with
+    the checkpoint. Or you can load state_dict in lazy mode ::
+
+        from deepspeed.utils.zero_to_fp32 import get_fp32_state_dict_from_zero_checkpoint
+        state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir, lazy_mode=True) # not on cpu
+        for name, lazy_tensor in state_dict.item():
+            tensor = lazy_tensor.contiguous()  # to cpu
+            print(name, tensor)
+            # del tensor to release memory if it no longer in use
+    """
+    if tag is None:
+        latest_path = os.path.join(checkpoint_dir, 'latest')
+        if os.path.isfile(latest_path):
+            with open(latest_path, 'r') as fd:
+                tag = fd.read().strip()
+        else:
+            raise ValueError(f"Unable to find 'latest' file at {latest_path}")
+
+    ds_checkpoint_dir = os.path.join(checkpoint_dir, tag)
+
+    if not os.path.isdir(ds_checkpoint_dir):
+        raise FileNotFoundError(f"Directory '{ds_checkpoint_dir}' doesn't exist")
+
+    state_dict = _get_fp32_state_dict_from_zero_checkpoint(ds_checkpoint_dir, exclude_frozen_parameters)
+    if lazy_mode:
+        return state_dict
+    else:
+        return to_torch_tensor(state_dict)
+
+
+def convert_zero_checkpoint_to_fp32_state_dict(checkpoint_dir,
+                                               output_dir,
+                                               max_shard_size="5GB",
+                                               safe_serialization=False,
+                                               tag=None,
+                                               exclude_frozen_parameters=False):
+    """
+    Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated ``state_dict`` file that can be
+    loaded with ``torch.load(file)`` + ``load_state_dict()`` and used for training without DeepSpeed.
+
+    Args:
+        - ``checkpoint_dir``: path to the desired checkpoint folder. (one that contains the tag-folder, like ``global_step14``)
+        - ``output_dir``: directory to the pytorch fp32 state_dict output files
+        - ``max_shard_size``: the maximum size for a checkpoint before being sharded, default value is 5GB
+        - ``safe_serialization``:  whether to save the model using `safetensors` or the traditional PyTorch way (that uses `pickle`).
+        - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in the file named ``latest`` in the checkpoint folder, e.g., ``global_step14``
+        - ``exclude_frozen_parameters``: exclude frozen parameters
+    """
+
+    # Dependency pre-check
+    if safe_serialization:
+        try:
+            from safetensors.torch import save_file
+        except ImportError:
+            print('If you want to use `safe_serialization`, please `pip install safetensors`')
+            raise
+    if max_shard_size is not None:
+        try:
+            from huggingface_hub import split_torch_state_dict_into_shards
+        except ImportError:
+            print('If you want to use `max_shard_size`, please `pip install huggingface_hub`')
+            raise
+
+    # Convert zero checkpoint to state_dict
+    state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir,
+                                                          tag,
+                                                          exclude_frozen_parameters,
+                                                          lazy_mode=True)
+
+    # Shard the model if it is too big.
+    weights_name = "model.safetensors" if safe_serialization else "pytorch_model.bin"
+    if max_shard_size is not None:
+        filename_pattern = weights_name.replace(".bin", "{suffix}.bin").replace(".safetensors", "{suffix}.safetensors")
+        # an memory-efficient approach for sharding
+        empty_state_dict = to_torch_tensor(state_dict, return_empty_tensor=True)
+        state_dict_split = split_torch_state_dict_into_shards(empty_state_dict,
+                                                              filename_pattern=filename_pattern,
+                                                              max_shard_size=max_shard_size)
+    else:
+        from collections import namedtuple
+        StateDictSplit = namedtuple("StateDictSplit", ["is_sharded", "filename_to_tensors"])
+        state_dict_split = StateDictSplit(is_sharded=False,
+                                          filename_to_tensors={weights_name: list(state_dict.keys())})
+
+    # Save the model by shard
+    os.makedirs(output_dir, exist_ok=True)
+    filename_to_tensors = state_dict_split.filename_to_tensors.items()
+    for shard_file, tensors in tqdm(filename_to_tensors, desc="Saving checkpoint shards"):
+        shard_state_dict = {tensor_name: state_dict[tensor_name] for tensor_name in tensors}
+        shard_state_dict = to_torch_tensor(shard_state_dict)
+        output_path = os.path.join(output_dir, shard_file)
+        if safe_serialization:
+            save_file(shard_state_dict, output_path, metadata={"format": "pt"})
+        else:
+            torch.save(shard_state_dict, output_path)
+        # release the memory of current shard
+        for tensor_name in list(shard_state_dict.keys()):
+            del state_dict[tensor_name]
+            del shard_state_dict[tensor_name]
+        del shard_state_dict
+        gc.collect()
+
+    # Save index if sharded
+    if state_dict_split.is_sharded:
+        index = {
+            "metadata": state_dict_split.metadata,
+            "weight_map": state_dict_split.tensor_to_filename,
+        }
+        save_index_file = "model.safetensors.index.json" if safe_serialization else "pytorch_model.bin.index.json"
+        save_index_file = os.path.join(output_dir, save_index_file)
+        with open(save_index_file, "w", encoding="utf-8") as f:
+            content = json.dumps(index, indent=2, sort_keys=True) + "\n"
+            f.write(content)
+
+
+def load_state_dict_from_zero_checkpoint(model, checkpoint_dir, tag=None):
+    """
+    1. Put the provided model to cpu
+    2. Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated ``state_dict``
+    3. Load it into the provided model
+
+    Args:
+        - ``model``: the model object to update
+        - ``checkpoint_dir``: path to the desired checkpoint folder. (one that contains the tag-folder, like ``global_step14``)
+        - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in the file named ``latest`` in the checkpoint folder, e.g., ``global_step14``
+
+    Returns:
+        - ``model`: modified model
+
+    Make sure you have plenty of CPU memory available before you call this function. If you don't
+    have enough use the ``zero_to_fp32.py`` utility to do the conversion. You will find it
+    conveniently placed for you in the checkpoint folder.
+
+    A typical usage might be ::
+
+        from deepspeed.utils.zero_to_fp32 import load_state_dict_from_zero_checkpoint
+        model = load_state_dict_from_zero_checkpoint(trainer.model, checkpoint_dir)
+        # submit to model hub or save the model to share with others
+
+    Note, that once this was run, the ``model`` will no longer be usable in the deepspeed context
+    of the same application. i.e. you will need to re-initialize the deepspeed engine, since
+    ``model.load_state_dict(state_dict)`` will remove all the deepspeed magic from it.
+
+    """
+    logger.info(f"Extracting fp32 weights")
+    state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir, tag)
+
+    logger.info(f"Overwriting model with fp32 weights")
+    model = model.cpu()
+    model.load_state_dict(state_dict, strict=False)
+
+    return model
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("checkpoint_dir",
+                        type=str,
+                        help="path to the desired checkpoint folder, e.g., path/checkpoint-12")
+    parser.add_argument("output_dir",
+                        type=str,
+                        help="directory to the pytorch fp32 state_dict output files"
+                        "(e.g. path/checkpoint-12-output/)")
+    parser.add_argument(
+        "--max_shard_size",
+        type=str,
+        default="5GB",
+        help="The maximum size for a checkpoint before being sharded. Checkpoints shard will then be each of size"
+        "lower than this size. If expressed as a string, needs to be digits followed by a unit (like `5MB`"
+        "We default it to 5GB in order for models to be able to run easily on free-tier google colab instances"
+        "without CPU OOM issues.")
+    parser.add_argument(
+        "--safe_serialization",
+        default=False,
+        action='store_true',
+        help="Whether to save the model using `safetensors` or the traditional PyTorch way (that uses `pickle`).")
+    parser.add_argument("-t",
+                        "--tag",
+                        type=str,
+                        default=None,
+                        help="checkpoint tag used as a unique identifier for checkpoint. e.g., global_step1")
+    parser.add_argument("--exclude_frozen_parameters", action='store_true', help="exclude frozen parameters")
+    parser.add_argument("-d", "--debug", action='store_true', help="enable debug")
+    args = parser.parse_args()
+
+    debug = args.debug
+
+    convert_zero_checkpoint_to_fp32_state_dict(args.checkpoint_dir,
+                                               args.output_dir,
+                                               max_shard_size=args.max_shard_size,
+                                               safe_serialization=args.safe_serialization,
+                                               tag=args.tag,
+                                               exclude_frozen_parameters=args.exclude_frozen_parameters)
diff --git a/te/baseline/data_15000_1000/eval_results.json b/te/baseline/data_15000_1000/eval_results.json
new file mode 100644
index 0000000000000000000000000000000000000000..341a146c15fb5aeb191160a18e15b9d08e691351
--- /dev/null
+++ b/te/baseline/data_15000_1000/eval_results.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0ecda65747f9ea7841f229ae3a5fc16b061439e5d52e16e95ed5b85cfa582e0
+size 266
diff --git a/te/baseline/data_15000_1000/special_tokens_map.json b/te/baseline/data_15000_1000/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..4ed2bd8240878a7a0d4fd2c60cdc89f6d7a5f1e1
--- /dev/null
+++ b/te/baseline/data_15000_1000/special_tokens_map.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83da1082df286d75a2984dc06ec439f4febc3d862ac55d199402e5d345f5773a
+size 372
diff --git a/te/baseline/data_15000_1000/tokenizer.json b/te/baseline/data_15000_1000/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..66cd9d7e0daec95eb10d16a63c615637dbbb7304
--- /dev/null
+++ b/te/baseline/data_15000_1000/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79e3e522635f3171300913bb421464a87de6222182a0570b9b2ccba2a964b2b4
+size 9085657
diff --git a/te/baseline/data_15000_1000/tokenizer_config.json b/te/baseline/data_15000_1000/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..7144ada11807e90b92529f17434f8d01915c3dff
--- /dev/null
+++ b/te/baseline/data_15000_1000/tokenizer_config.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d967a51bb800d3e471ea23dd3f7b054b136600238bbbb40612b8b96b0370746e
+size 55427
diff --git a/te/baseline/data_15000_1000/train.log b/te/baseline/data_15000_1000/train.log
new file mode 100644
index 0000000000000000000000000000000000000000..fa47decfce777d4732aef91dd1e8aae37f419345
--- /dev/null
+++ b/te/baseline/data_15000_1000/train.log
@@ -0,0 +1,2150 @@
+W0626 22:15:26.797451 1357899 site-packages/torch/distributed/run.py:766] 
+W0626 22:15:26.797451 1357899 site-packages/torch/distributed/run.py:766] *****************************************
+W0626 22:15:26.797451 1357899 site-packages/torch/distributed/run.py:766] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+W0626 22:15:26.797451 1357899 site-packages/torch/distributed/run.py:766] *****************************************
+[2025-06-26 22:15:32,611] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-26 22:15:32,688] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-26 22:15:32,698] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-26 22:15:32,702] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-26 22:15:34,032] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-26 22:15:34,157] [INFO] [comm.py:675:init_distributed] cdb=None
+[2025-06-26 22:15:34,191] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-26 22:15:34,195] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-26 22:15:34,209] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
+[2025-06-26 22:15:34,317] [INFO] [comm.py:675:init_distributed] cdb=None
+[2025-06-26 22:15:34,317] [INFO] [comm.py:706:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
+[2025-06-26 22:15:34,324] [INFO] [comm.py:675:init_distributed] cdb=None
+[2025-06-26 22:15:34,339] [INFO] [comm.py:675:init_distributed] cdb=None
+06/26/2025 22:15:34 - WARNING - __main__ - Process rank: 3, device: cuda:3, n_gpu: 1distributed training: True, 16-bits training: False
+06/26/2025 22:15:34 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1distributed training: True, 16-bits training: False
+06/26/2025 22:15:34 - INFO - __main__ - Training/evaluation parameters LoRATrainingArguments(
+_n_gpu=1,
+accelerator_config={'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None, 'use_configured_state': False},
+adafactor=False,
+adam_beta1=0.9,
+adam_beta2=0.999,
+adam_epsilon=1e-08,
+auto_find_batch_size=False,
+batch_eval_metrics=False,
+bf16=True,
+bf16_full_eval=True,
+data_seed=None,
+dataloader_drop_last=False,
+dataloader_num_workers=2,
+dataloader_persistent_workers=False,
+dataloader_pin_memory=True,
+dataloader_prefetch_factor=None,
+ddp_backend=None,
+ddp_broadcast_buffers=None,
+ddp_bucket_cap_mb=None,
+ddp_find_unused_parameters=None,
+ddp_timeout=3600,
+debug=[],
+deepspeed=./config/deepspeed_config.json,
+disable_tqdm=False,
+dispatch_batches=None,
+do_eval=True,
+do_predict=False,
+do_train=True,
+eval_accumulation_steps=None,
+eval_delay=0,
+eval_do_concat_batches=True,
+eval_on_start=True,
+eval_steps=200,
+eval_strategy=steps,
+eval_use_gather_object=False,
+evaluation_strategy=None,
+fp16=False,
+fp16_backend=auto,
+fp16_full_eval=False,
+fp16_opt_level=O1,
+fsdp=[],
+fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False},
+fsdp_min_num_params=0,
+fsdp_transformer_layer_cls_to_wrap=None,
+full_determinism=False,
+gradient_accumulation_steps=1,
+gradient_checkpointing=True,
+gradient_checkpointing_kwargs=None,
+greater_is_better=False,
+group_by_length=False,
+half_precision_backend=auto,
+hub_always_push=False,
+hub_model_id=None,
+hub_private_repo=False,
+hub_strategy=every_save,
+hub_token=<HUB_TOKEN>,
+ignore_data_skip=False,
+include_inputs_for_metrics=False,
+include_num_input_tokens_seen=False,
+include_tokens_per_second=False,
+jit_mode_eval=False,
+label_names=None,
+label_smoothing_factor=0.0,
+learning_rate=0.0005,
+length_column_name=length,
+load_best_model_at_end=True,
+load_lora_from=None,
+local_rank=0,
+log_level=passive,
+log_level_replica=warning,
+log_on_each_node=True,
+logging_dir=./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/runs/Jun26_22-15-32_innmi1srh2-p040,
+logging_first_step=False,
+logging_nan_inf_filter=True,
+logging_steps=1.0,
+logging_strategy=steps,
+lora_config=./config/lora_config.json,
+lr_scheduler_kwargs={},
+lr_scheduler_type=inverse_sqrt,
+max_grad_norm=1.0,
+max_steps=-1,
+metric_for_best_model=eval_loss,
+mp_parameters=,
+neftune_noise_alpha=None,
+no_cuda=False,
+num_train_epochs=5.0,
+optim=adamw_torch,
+optim_args=None,
+optim_target_modules=None,
+output_dir=./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/,
+overwrite_output_dir=True,
+past_index=-1,
+per_device_eval_batch_size=25,
+per_device_train_batch_size=25,
+prediction_loss_only=False,
+push_to_hub=False,
+push_to_hub_model_id=None,
+push_to_hub_organization=None,
+push_to_hub_token=<PUSH_TO_HUB_TOKEN>,
+ray_scope=last,
+remove_unused_columns=True,
+report_to=['wandb'],
+restore_callback_states_from_checkpoint=False,
+resume_from_checkpoint=None,
+run_name=./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/,
+save_on_each_node=False,
+save_only_model=False,
+save_safetensors=True,
+save_steps=200,
+save_strategy=steps,
+save_total_limit=1,
+seed=1,
+skip_memory_metrics=True,
+split_batches=None,
+tf32=None,
+torch_compile=False,
+torch_compile_backend=None,
+torch_compile_mode=None,
+torch_empty_cache_steps=None,
+torchdynamo=None,
+tpu_metrics_debug=False,
+tpu_num_cores=None,
+use_cpu=False,
+use_int8_training=False,
+use_ipex=False,
+use_legacy_prediction_loop=False,
+use_lora=True,
+use_mps_device=False,
+warmup_ratio=0.03,
+warmup_steps=0,
+weight_decay=0.0,
+)
+06/26/2025 22:15:34 - WARNING - __main__ - Process rank: 1, device: cuda:1, n_gpu: 1distributed training: True, 16-bits training: False
+06/26/2025 22:15:34 - WARNING - __main__ - Process rank: 2, device: cuda:2, n_gpu: 1distributed training: True, 16-bits training: False
+Using custom data configuration default-5191cedfd0d54773
+06/26/2025 22:15:35 - INFO - datasets.builder - Using custom data configuration default-5191cedfd0d54773
+Loading Dataset Infos from /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/datasets/packaged_modules/json
+06/26/2025 22:15:35 - INFO - datasets.info - Loading Dataset Infos from /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/datasets/packaged_modules/json
+Overwrite dataset info from restored data version if exists.
+06/26/2025 22:15:35 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists.
+Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-5191cedfd0d54773/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+06/26/2025 22:15:35 - INFO - datasets.info - Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-5191cedfd0d54773/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+Found cached dataset json (/home/iitm_admin/.cache/huggingface/datasets/json/default-5191cedfd0d54773/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092)
+06/26/2025 22:15:35 - INFO - datasets.builder - Found cached dataset json (/home/iitm_admin/.cache/huggingface/datasets/json/default-5191cedfd0d54773/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092)
+Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-5191cedfd0d54773/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+06/26/2025 22:15:35 - INFO - datasets.info - Loading Dataset info from /home/iitm_admin/.cache/huggingface/datasets/json/default-5191cedfd0d54773/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092
+[INFO|configuration_utils.py:733] 2025-06-26 22:15:35,637 >> loading configuration file config.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/config.json
+[INFO|configuration_utils.py:821] 2025-06-26 22:15:35,638 >> Model config LlamaConfig {
+  "_name_or_path": "meta-llama/Llama-3.1-8B-Instruct",
+  "additional_loss_layer": 16,
+  "alignment_matrices_path": null,
+  "apply_inverse": false,
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "contrastive_loss_temperature": 1.0,
+  "contrastive_loss_weight": 1.0,
+  "contrastive_pooling_type": "mean",
+  "distance_function": "cosine",
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "inject_Ws": false,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "only_train_contrastive": false,
+  "only_train_language_modeling": true,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.44.0.dev0",
+  "unidirectional_contrastive_loss": false,
+  "use_cache": true,
+  "vocab_size": 128256
+}
+
+[INFO|tokenization_utils_base.py:2269] 2025-06-26 22:15:35,875 >> loading file tokenizer.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/tokenizer.json
+[INFO|tokenization_utils_base.py:2269] 2025-06-26 22:15:35,875 >> loading file added_tokens.json from cache at None
+[INFO|tokenization_utils_base.py:2269] 2025-06-26 22:15:35,875 >> loading file special_tokens_map.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/special_tokens_map.json
+[INFO|tokenization_utils_base.py:2269] 2025-06-26 22:15:35,875 >> loading file tokenizer_config.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2513] 2025-06-26 22:15:36,141 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
+06/26/2025 22:15:36 - INFO - __main__ - Tokenizer is fast: True
+[INFO|modeling_utils.py:3667] 2025-06-26 22:15:36,144 >> loading weights file model.safetensors from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/model.safetensors.index.json
+[INFO|modeling_utils.py:1591] 2025-06-26 22:15:36,145 >> Instantiating LlamaForCausalLM model under default dtype torch.bfloat16.
+[WARNING|logging.py:328] 2025-06-26 22:15:36,147 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+[INFO|configuration_utils.py:1038] 2025-06-26 22:15:36,149 >> Generate config GenerationConfig {
+  "bos_token_id": 128000,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ]
+}
+
+[WARNING|logging.py:328] 2025-06-26 22:15:36,219 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s][WARNING|logging.py:328] 2025-06-26 22:15:36,365 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  4.29it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  4.61it/s][WARNING|logging.py:328] 2025-06-26 22:15:36,590 >> You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  5.00it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  4.89it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  5.10it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  6.32it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  5.34it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  5.13it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  6.72it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:00,  3.69it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.59it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.34it/s]
+[INFO|modeling_utils.py:4499] 2025-06-26 22:15:36,973 >> All model checkpoint weights were used when initializing LlamaForCausalLM.
+
+[INFO|modeling_utils.py:4507] 2025-06-26 22:15:36,973 >> All the weights of LlamaForCausalLM were initialized from the model checkpoint at meta-llama/Llama-3.1-8B-Instruct.
+If your task is similar to the task the model of the checkpoint was trained on, you can already use LlamaForCausalLM for predictions without further training.
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  6.88it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  6.58it/s]
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.46it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.28it/s]
+
Loading checkpoint shards:  50%|█████     | 2/4 [00:00<00:00,  4.53it/s][INFO|configuration_utils.py:993] 2025-06-26 22:15:37,189 >> loading configuration file generation_config.json from cache at /home/iitm_admin/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/generation_config.json
+[INFO|configuration_utils.py:1038] 2025-06-26 22:15:37,189 >> Generate config GenerationConfig {
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "temperature": 0.6,
+  "top_p": 0.9
+}
+
+adding special tokens...
+06/26/2025 22:15:37 - INFO - __main__ - ================ pad, eos, bos, unk, padding ================ 
+06/26/2025 22:15:37 - INFO - __main__ - <|eot_id|>, 128009
+06/26/2025 22:15:37 - INFO - __main__ - <|eot_id|>, 128009
+06/26/2025 22:15:37 - INFO - __main__ - <|begin_of_text|>, 128000
+06/26/2025 22:15:37 - INFO - __main__ - <|reserved_special_token_0|>, 128002
+06/26/2025 22:15:37 - INFO - __main__ - right
+06/26/2025 22:15:37 - INFO - __main__ - lora_r : 8
+06/26/2025 22:15:37 - INFO - __main__ - lora_alpha : 16
+06/26/2025 22:15:37 - INFO - __main__ - lora_dropout : 0.1
+06/26/2025 22:15:37 - INFO - __main__ - lora_target_modules : ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj']
+06/26/2025 22:15:37 - INFO - __main__ - LoRA configs: LoraConfig(task_type='CAUSAL_LM', peft_type=<PeftType.LORA: 'LORA'>, auto_mapping=None, base_model_name_or_path=None, revision=None, inference_mode=False, r=8, target_modules={'q_proj', 'gate_proj', 'o_proj', 'up_proj', 'down_proj', 'v_proj', 'k_proj'}, exclude_modules=None, lora_alpha=16, lora_dropout=0.1, fan_in_fan_out=False, bias='none', use_rslora=False, modules_to_save=None, init_lora_weights=True, layers_to_transform=None, layers_pattern=None, rank_pattern={}, alpha_pattern={}, megatron_config=None, megatron_core='megatron.core', trainable_token_indices=None, loftq_config={}, eva_config=None, corda_config=None, use_dora=False, layer_replication=None, runtime_config=LoraRuntimeConfig(ephemeral_gpu_offload=False), lora_bias=False)
+adding special tokens...
+
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:00<00:00,  5.31it/s]adding special tokens...
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  6.06it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00,  5.44it/s]
+trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+06/26/2025 22:15:37 - INFO - __main__ - block size: 2048
+trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank3]:[W626 22:15:37.319043922 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 3]  using GPU 3 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-5191cedfd0d54773/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-bd389143e9c8afb6.arrow
+06/26/2025 22:15:37 - INFO - datasets.arrow_dataset - Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-5191cedfd0d54773/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-bd389143e9c8afb6.arrow
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank1]:[W626 22:15:37.347307627 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 1]  using GPU 1 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+adding special tokens...
+Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-5191cedfd0d54773/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-99dfc9d7bb588664.arrow
+06/26/2025 22:15:37 - INFO - datasets.arrow_dataset - Loading cached processed dataset at /home/iitm_admin/.cache/huggingface/datasets/json/default-5191cedfd0d54773/0.0.0/f4e89e8750d5d5ffbef2c078bf0ddfedef29dc2faff52a6255cf513c05eb1092/cache-99dfc9d7bb588664.arrow
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank0]:[W626 22:15:37.590660491 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 0]  using GPU 0 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
+PeftModelForCausalLM(
+  (base_model): LoraModel(
+    (model): LlamaForCausalLM(
+      (model): LlamaModel(
+        (embed_tokens): Embedding(128256, 4096)
+        (layers): ModuleList(
+          (0-31): 32 x LlamaDecoderLayer(
+            (self_attn): LlamaFlashAttention2(
+              (q_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (k_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (v_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=1024, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=1024, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (o_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (rotary_emb): LlamaRotaryEmbedding()
+            )
+            (mlp): LlamaMLP(
+              (gate_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (up_proj): lora.Linear(
+                (base_layer): Linear(in_features=4096, out_features=14336, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=4096, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=14336, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (down_proj): lora.Linear(
+                (base_layer): Linear(in_features=14336, out_features=4096, bias=False)
+                (lora_dropout): ModuleDict(
+                  (default): Dropout(p=0.1, inplace=False)
+                )
+                (lora_A): ModuleDict(
+                  (default): Linear(in_features=14336, out_features=8, bias=False)
+                )
+                (lora_B): ModuleDict(
+                  (default): Linear(in_features=8, out_features=4096, bias=False)
+                )
+                (lora_embedding_A): ParameterDict()
+                (lora_embedding_B): ParameterDict()
+                (lora_magnitude_vector): ModuleDict()
+              )
+              (act_fn): SiLU()
+            )
+            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
+          )
+        )
+        (norm): LlamaRMSNorm((4096,), eps=1e-05)
+        (rotary_emb): LlamaRotaryEmbedding()
+      )
+      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
+    )
+  )
+)
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[rank2]:[W626 22:15:37.725436811 ProcessGroupNCCL.cpp:4718] [PG ID 0 PG GUID 0 Rank 2]  using GPU 2 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device.
+06/26/2025 22:15:39 - INFO - __main__ - xxx: Showcase the tokenized training samples.
+{'input_ids': [128000, 128006, 9125, 128007, 271, 32405, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 245, 32405, 96, 32405, 123, 32405, 97, 94355, 116, 32405, 106, 32405, 116, 53898, 235, 32405, 107, 32405, 110, 32405, 101, 53898, 223, 94355, 103, 32405, 108, 32405, 123, 32405, 115, 53898, 235, 32405, 243, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 32405, 110, 53898, 233, 94355, 101, 32405, 123, 32405, 103, 53898, 223, 32405, 96, 53898, 223, 32405, 110, 53898, 230, 32405, 101, 94355, 116, 32405, 117, 32405, 122, 32405, 107, 32405, 243, 53898, 223, 32405, 94, 32405, 123, 94355, 103, 32405, 122, 32405, 97, 53898, 235, 32405, 108, 32405, 110, 53898, 233, 94355, 231, 32405, 101, 53898, 235, 32405, 101, 32405, 122, 32405, 108, 53898, 223, 13, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 228, 32405, 110, 53898, 233, 32405, 248, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 99, 32405, 114, 32405, 101, 53898, 223, 94355, 116, 53898, 235, 32405, 103, 32405, 115, 53898, 235, 32405, 253, 32405, 224, 32405, 245, 32405, 122, 94355, 113, 32405, 123, 32405, 113, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 330, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 3047, 94355, 227, 32405, 101, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 122, 32405, 108, 32405, 224, 32405, 255, 32405, 123, 32405, 224, 32405, 248, 32405, 123, 11, 94355, 110, 53898, 228, 32405, 243, 53898, 235, 32405, 243, 32405, 110, 53898, 223, 94355, 97, 32405, 108, 53898, 235, 32405, 243, 32405, 105, 32405, 99, 53898, 235, 32405, 100, 32405, 224, 32405, 245, 32405, 122, 94355, 248, 53898, 229, 32405, 107, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 248, 32405, 123, 32405, 113, 32405, 108, 32405, 245, 32405, 122, 330, 32405, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 1630, 1210, 94355, 227, 32405, 101, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 97, 53898, 228, 32405, 110, 32405, 103, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 229, 32405, 243, 53898, 235, 32405, 243, 32405, 94, 1630, 94355, 227, 32405, 101, 53898, 229, 32405, 99, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 13, 128009, 198, 128006, 882, 128007, 271, 32405, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 25, 220, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 17, 11, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 400, 18, 11, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 228, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 87, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 94355, 240, 32405, 243, 53898, 235, 32405, 243, 53898, 232, 32405, 243, 53898, 235, 32405, 243, 32405, 253, 32405, 123, 400, 16, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 97, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 243, 53898, 232, 32405, 224, 32405, 97, 32405, 106, 32405, 224, 32405, 99, 32405, 123, 94355, 116, 53898, 235, 32405, 101, 53898, 229, 32405, 117, 32405, 123, 32405, 97, 53898, 223, 32405, 110, 32405, 243, 53898, 223, 94355, 243, 53898, 232, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 94, 53898, 235, 32405, 108, 32405, 123, 32405, 224, 32405, 243, 53898, 235, 32405, 116, 53898, 235, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 32405, 101, 53898, 223, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 94355, 243, 53898, 233, 32405, 116, 32405, 224, 94355, 228, 32405, 106, 53898, 228, 94355, 236, 32405, 224, 32405, 97, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 32405, 101, 53898, 223, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 223, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 30, 94355, 103, 53898, 230, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 18, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 107, 32405, 101, 32405, 123, 94355, 113, 53898, 229, 32405, 108, 32405, 123, 32405, 107, 32405, 105, 53898, 223, 32405, 110, 53898, 235, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 94355, 236, 32405, 224, 32405, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 32405, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 245, 32405, 96, 32405, 123, 32405, 97, 94355, 116, 32405, 106, 32405, 116, 53898, 235, 32405, 107, 32405, 110, 32405, 101, 53898, 223, 94355, 103, 32405, 108, 32405, 123, 32405, 115, 53898, 235, 32405, 243, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 32405, 110, 53898, 233, 94355, 101, 32405, 123, 32405, 103, 53898, 223, 32405, 96, 53898, 223, 32405, 110, 53898, 230, 32405, 101, 94355, 116, 32405, 117, 32405, 122, 32405, 107, 32405, 243, 53898, 223, 32405, 94, 32405, 123, 94355, 103, 32405, 122, 32405, 97, 53898, 235, 32405, 108, 32405, 110, 53898, 233, 94355, 231, 32405, 101, 53898, 235, 32405, 101, 32405, 122, 32405, 108, 53898, 223, 13, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 228, 32405, 110, 53898, 233, 32405, 248, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 99, 32405, 114, 32405, 101, 53898, 223, 94355, 116, 53898, 235, 32405, 103, 32405, 115, 53898, 235, 32405, 253, 32405, 224, 32405, 245, 32405, 122, 94355, 113, 32405, 123, 32405, 113, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 330, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 3047, 94355, 227, 32405, 101, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 122, 32405, 108, 32405, 224, 32405, 255, 32405, 123, 32405, 224, 32405, 248, 32405, 123, 11, 94355, 110, 53898, 228, 32405, 243, 53898, 235, 32405, 243, 32405, 110, 53898, 223, 94355, 97, 32405, 108, 53898, 235, 32405, 243, 32405, 105, 32405, 99, 53898, 235, 32405, 100, 32405, 224, 32405, 245, 32405, 122, 94355, 248, 53898, 229, 32405, 107, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 248, 32405, 123, 32405, 113, 32405, 108, 32405, 245, 32405, 122, 330, 32405, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 1630, 1210, 94355, 227, 32405, 101, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 97, 53898, 228, 32405, 110, 32405, 103, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 229, 32405, 243, 53898, 235, 32405, 243, 32405, 94, 1630, 94355, 227, 32405, 101, 53898, 229, 32405, 99, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 13, 128009, 198, 128006, 882, 128007, 271, 32405, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 25, 220, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 17, 11, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 400, 18, 11, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 228, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 87, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 94355, 240, 32405, 243, 53898, 235, 32405, 243, 53898, 232, 32405, 243, 53898, 235, 32405, 243, 32405, 253, 32405, 123, 400, 16, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 97, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 243, 53898, 232, 32405, 224, 32405, 97, 32405, 106, 32405, 224, 32405, 99, 32405, 123, 94355, 116, 53898, 235, 32405, 101, 53898, 229, 32405, 117, 32405, 123, 32405, 97, 53898, 223, 32405, 110, 32405, 243, 53898, 223, 94355, 243, 53898, 232, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 94, 53898, 235, 32405, 108, 32405, 123, 32405, 224, 32405, 243, 53898, 235, 32405, 116, 53898, 235, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 32405, 101, 53898, 223, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 94355, 243, 53898, 233, 32405, 116, 32405, 224, 94355, 228, 32405, 106, 53898, 228, 94355, 236, 32405, 224, 32405, 97, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 32405, 101, 53898, 223, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 223, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 30, 94355, 103, 53898, 230, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 18, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 107, 32405, 101, 32405, 123, 94355, 113, 53898, 229, 32405, 108, 32405, 123, 32405, 107, 32405, 105, 53898, 223, 32405, 110, 53898, 235, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 94355, 236, 32405, 224, 32405, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 32405, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 245, 32405, 96, 32405, 123, 32405, 97, 94355, 116, 32405, 106, 32405, 116, 53898, 235, 32405, 107, 32405, 110, 32405, 101, 53898, 223, 94355, 103, 32405, 108, 32405, 123, 32405, 115, 53898, 235, 32405, 243, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 32405, 110, 53898, 233, 94355, 101, 32405, 123, 32405, 103, 53898, 223, 32405, 96, 53898, 223, 32405, 110, 53898, 230, 32405, 101, 94355, 116, 32405, 117, 32405, 122, 32405, 107, 32405, 243, 53898, 223, 32405, 94, 32405, 123, 94355, 103, 32405, 122, 32405, 97, 53898, 235, 32405, 108, 32405, 110, 53898, 233, 94355, 231, 32405, 101, 53898, 235, 32405, 101, 32405, 122, 32405, 108, 53898, 223, 13, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 228, 32405, 110, 53898, 233, 32405, 248, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 99, 32405, 114, 32405, 101, 53898, 223, 94355, 116, 53898, 235, 32405, 103, 32405, 115, 53898, 235, 32405, 253, 32405, 224, 32405, 245, 32405, 122, 94355, 113, 32405, 123, 32405, 113, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 330, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 3047, 94355, 227, 32405, 101, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 122, 32405, 108, 32405, 224, 32405, 255, 32405, 123, 32405, 224, 32405, 248, 32405, 123, 11, 94355, 110, 53898, 228, 32405, 243, 53898, 235, 32405, 243, 32405, 110, 53898, 223, 94355, 97, 32405, 108, 53898, 235, 32405, 243, 32405, 105, 32405, 99, 53898, 235, 32405, 100, 32405, 224, 32405, 245, 32405, 122, 94355, 248, 53898, 229, 32405, 107, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 248, 32405, 123, 32405, 113, 32405, 108, 32405, 245, 32405, 122, 330, 32405, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 1630, 1210, 94355, 227, 32405, 101, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 97, 53898, 228, 32405, 110, 32405, 103, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 229, 32405, 243, 53898, 235, 32405, 243, 32405, 94, 1630, 94355, 227, 32405, 101, 53898, 229, 32405, 99, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 13, 128009, 198, 128006, 882, 128007, 271, 32405, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 25, 220, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 17, 11, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 400, 18, 11, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 228, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 87, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 94355, 240, 32405, 243, 53898, 235, 32405, 243, 53898, 232, 32405, 243, 53898, 235, 32405, 243, 32405, 253, 32405, 123, 400, 16, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 97, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 243, 53898, 232, 32405, 224, 32405, 97, 32405, 106, 32405, 224, 32405, 99, 32405, 123, 94355, 116, 53898, 235, 32405, 101, 53898, 229, 32405, 117, 32405, 123, 32405, 97, 53898, 223, 32405, 110, 32405, 243, 53898, 223, 94355, 243, 53898, 232, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 94, 53898, 235, 32405, 108, 32405, 123, 32405, 224, 32405, 243, 53898, 235, 32405, 116, 53898, 235, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 32405, 101, 53898, 223, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 94355, 243, 53898, 233, 32405, 116, 32405, 224, 94355, 228, 32405, 106, 53898, 228, 94355, 236, 32405, 224, 32405, 97, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 32405, 101, 53898, 223, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 223, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 30, 94355, 103, 53898, 230, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 18, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 107, 32405, 101, 32405, 123, 94355, 113, 53898, 229, 32405, 108, 32405, 123, 32405, 107, 32405, 105, 53898, 223, 32405, 110, 53898, 235, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 94355, 236, 32405, 224, 32405, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 32405, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 245, 32405, 96, 32405, 123, 32405, 97, 94355, 116, 32405, 106, 32405, 116, 53898, 235, 32405, 107, 32405, 110, 32405, 101, 53898, 223, 94355, 103, 32405, 108, 32405, 123, 32405, 115, 53898, 235, 32405, 243, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 32405, 110, 53898, 233, 94355, 101, 32405, 123, 32405, 103, 53898, 223, 32405, 96, 53898, 223, 32405, 110, 53898, 230, 32405, 101, 94355, 116, 32405, 117, 32405, 122, 32405, 107, 32405, 243, 53898, 223, 32405, 94, 32405, 123, 94355, 103, 32405, 122, 32405, 97, 53898, 235, 32405, 108, 32405, 110, 53898, 233, 94355, 231, 32405, 101, 53898, 235, 32405, 101, 32405, 122, 32405, 108, 53898, 223, 13, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 228, 32405, 110, 53898, 233, 32405, 248, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 99, 32405, 114, 32405, 101, 53898, 223, 94355, 116, 53898, 235, 32405, 103, 32405, 115, 53898, 235, 32405, 253, 32405, 224, 32405, 245, 32405, 122, 94355, 113, 32405, 123, 32405, 113, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 330, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 3047, 94355, 227, 32405, 101, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 122, 32405, 108, 32405, 224, 32405, 255, 32405, 123, 32405, 224, 32405, 248, 32405, 123, 11, 94355, 110, 53898, 228, 32405, 243, 53898, 235, 32405, 243, 32405, 110, 53898, 223, 94355, 97, 32405, 108, 53898, 235, 32405, 243, 32405, 105, 32405, 99, 53898, 235, 32405, 100, 32405, 224, 32405, 245, 32405, 122, 94355, 248, 53898, 229, 32405, 107, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 248, 32405, 123, 32405, 113, 32405, 108, 32405, 245, 32405, 122, 330, 32405, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 1630, 1210, 94355, 227, 32405, 101, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 97, 53898, 228, 32405, 110, 32405, 103, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 229, 32405, 243, 53898, 235, 32405, 243, 32405, 94, 1630, 94355, 227, 32405, 101, 53898, 229, 32405, 99, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 13, 128009, 198, 128006, 882, 128007, 271, 32405, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 25, 220, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 17, 11, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 400, 18, 11, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 228, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 87, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 94355, 240, 32405, 243, 53898, 235, 32405, 243, 53898, 232, 32405, 243, 53898, 235, 32405, 243, 32405, 253, 32405, 123, 400, 16, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 97, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 243, 53898, 232, 32405, 224, 32405, 97, 32405, 106, 32405, 224, 32405, 99, 32405, 123, 94355, 116, 53898, 235, 32405, 101, 53898, 229, 32405, 117, 32405, 123, 32405, 97, 53898, 223, 32405, 110, 32405, 243, 53898, 223, 94355, 243, 53898, 232, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 94, 53898, 235, 32405, 108, 32405, 123, 32405, 224, 32405, 243, 53898, 235, 32405, 116, 53898, 235, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 32405, 101, 53898, 223, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 94355, 243, 53898, 233, 32405, 116, 32405, 224, 94355, 228, 32405, 106, 53898, 228, 94355, 236, 32405, 224, 32405, 97, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 32405, 101, 53898, 223, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 223, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 30, 94355, 103, 53898, 230, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 18, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 107, 32405, 101, 32405, 123, 94355, 113, 53898, 229, 32405, 108, 32405, 123, 32405, 107, 32405, 105, 53898, 223, 32405, 110, 53898, 235, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 94355, 236, 32405, 224, 32405, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 32405, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 245, 32405, 96, 32405, 123, 32405, 97, 94355, 116, 32405, 106, 32405, 116, 53898, 235, 32405, 107, 32405, 110, 32405, 101, 53898, 223, 94355, 103, 32405, 108, 32405, 123, 32405, 115, 53898, 235, 32405, 243, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 32405, 110, 53898, 233, 94355, 101, 32405, 123, 32405, 103, 53898, 223, 32405, 96, 53898, 223, 32405, 110, 53898, 230, 32405, 101, 94355, 116, 32405, 117, 32405, 122, 32405, 107, 32405, 243, 53898, 223, 32405, 94, 32405, 123, 94355, 103, 32405, 122, 32405, 97, 53898, 235, 32405, 108, 32405, 110, 53898, 233, 94355, 231, 32405, 101, 53898, 235, 32405, 101, 32405, 122, 32405, 108, 53898, 223, 13, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 228, 32405, 110, 53898, 233, 32405, 248, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 99, 32405, 114, 32405, 101, 53898, 223, 94355, 116, 53898, 235, 32405, 103, 32405, 115, 53898, 235, 32405, 253, 32405, 224, 32405, 245, 32405, 122, 94355, 113, 32405, 123, 32405, 113, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 330, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 3047, 94355, 227, 32405, 101, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 122, 32405, 108, 32405, 224, 32405, 255, 32405, 123, 32405, 224, 32405, 248, 32405, 123, 11, 94355, 110, 53898, 228, 32405, 243, 53898, 235, 32405, 243, 32405, 110, 53898, 223, 94355, 97, 32405, 108, 53898, 235, 32405, 243, 32405, 105, 32405, 99, 53898, 235, 32405, 100, 32405, 224, 32405, 245, 32405, 122, 94355, 248, 53898, 229, 32405, 107, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 248, 32405, 123, 32405, 113, 32405, 108, 32405, 245, 32405, 122, 330, 32405, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 1630, 1210, 94355, 227, 32405, 101, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 97, 53898, 228, 32405, 110, 32405, 103, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 229, 32405, 243, 53898, 235, 32405, 243, 32405, 94, 1630, 94355, 227, 32405, 101, 53898, 229, 32405, 99, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 13, 128009, 198, 128006, 882, 128007, 271, 32405, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 25, 220, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 17, 11, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 400, 18, 11, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 228, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 87, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 94355, 240, 32405, 243, 53898, 235, 32405, 243, 53898, 232, 32405, 243, 53898, 235, 32405, 243, 32405, 253, 32405, 123, 400, 16, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 97, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 243, 53898, 232, 32405, 224, 32405, 97, 32405, 106, 32405, 224, 32405, 99, 32405, 123, 94355, 116, 53898, 235, 32405, 101, 53898, 229, 32405, 117, 32405, 123, 32405, 97, 53898, 223, 32405, 110, 32405, 243, 53898, 223, 94355, 243, 53898, 232, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 94, 53898, 235, 32405, 108, 32405, 123, 32405, 224, 32405, 243, 53898, 235, 32405, 116, 53898, 235, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 32405, 101, 53898, 223, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 94355, 243, 53898, 233, 32405, 116, 32405, 224, 94355, 228, 32405, 106, 53898, 228, 94355, 236, 32405, 224, 32405, 97, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 32405, 101, 53898, 223, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 223, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 30, 94355, 103, 53898, 230, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 18, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 107, 32405, 101, 32405, 123, 94355, 113, 53898, 229, 32405, 108, 32405, 123, 32405, 107, 32405, 105, 53898, 223, 32405, 110, 53898, 235, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 94355, 236, 32405, 224, 32405, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 32405, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 245, 32405, 96, 32405, 123, 32405, 97, 94355, 116, 32405, 106, 32405, 116, 53898, 235, 32405, 107, 32405, 110, 32405, 101, 53898, 223, 94355, 103, 32405, 108, 32405, 123, 32405, 115, 53898, 235, 32405, 243, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 32405, 110, 53898, 233, 94355, 101, 32405, 123, 32405, 103, 53898, 223, 32405, 96, 53898, 223, 32405, 110, 53898, 230, 32405, 101, 94355, 116, 32405, 117, 32405, 122, 32405, 107, 32405, 243, 53898, 223, 32405, 94, 32405, 123, 94355, 103, 32405, 122, 32405, 97, 53898, 235, 32405, 108, 32405, 110, 53898, 233, 94355, 231, 32405, 101, 53898, 235, 32405, 101, 32405, 122, 32405, 108, 53898, 223, 13, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 228, 32405, 110, 53898, 233, 32405, 248, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 99, 32405, 114, 32405, 101, 53898, 223, 94355, 116, 53898, 235, 32405, 103, 32405, 115, 53898, 235, 32405, 253, 32405, 224, 32405, 245, 32405, 122, 94355, 113, 32405, 123, 32405, 113, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 330, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 3047, 94355, 227, 32405, 101, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 122, 32405, 108, 32405, 224, 32405, 255, 32405, 123, 32405, 224, 32405, 248, 32405, 123, 11, 94355, 110, 53898, 228, 32405, 243, 53898, 235, 32405, 243, 32405, 110, 53898, 223, 94355, 97, 32405, 108, 53898, 235, 32405, 243, 32405, 105, 32405, 99, 53898, 235, 32405, 100, 32405, 224, 32405, 245, 32405, 122, 94355, 248, 53898, 229, 32405, 107, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 248, 32405, 123, 32405, 113, 32405, 108, 32405, 245, 32405, 122, 330, 32405, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 1630, 1210, 94355, 227, 32405, 101, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 97, 53898, 228, 32405, 110, 32405, 103, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 229, 32405, 243, 53898, 235, 32405, 243, 32405, 94, 1630, 94355, 227, 32405, 101, 53898, 229, 32405, 99, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 13, 128009, 198, 128006, 882, 128007, 271, 32405, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 25, 220, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 17, 11, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 400, 18, 11, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 228, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 87, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 94355, 240, 32405, 243, 53898, 235, 32405, 243, 53898, 232, 32405, 243, 53898, 235, 32405, 243, 32405, 253, 32405, 123, 400, 16, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 97, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 243, 53898, 232, 32405, 224, 32405, 97, 32405, 106, 32405, 224, 32405, 99, 32405, 123, 94355, 116, 53898, 235, 32405, 101, 53898, 229, 32405, 117, 32405, 123, 32405, 97, 53898, 223, 32405, 110, 32405, 243, 53898, 223, 94355, 243, 53898, 232, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 94, 53898, 235, 32405, 108, 32405, 123, 32405, 224, 32405, 243, 53898, 235, 32405, 116, 53898, 235, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 32405, 101, 53898, 223, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 94355, 243, 53898, 233, 32405, 116, 32405, 224, 94355, 228, 32405, 106, 53898, 228, 94355, 236, 32405, 224, 32405, 97, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 32405, 101, 53898, 223, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 223, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 30, 94355, 103, 53898, 230, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 18, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 107, 32405, 101, 32405, 123, 94355, 113, 53898, 229, 32405, 108, 32405, 123, 32405, 107, 32405, 105, 53898, 223, 32405, 110, 53898, 235, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 94355, 236, 32405, 224, 32405, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009]}{'input_ids': [128000, 128006, 9125, 128007, 271, 32405, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 245, 32405, 96, 32405, 123, 32405, 97, 94355, 116, 32405, 106, 32405, 116, 53898, 235, 32405, 107, 32405, 110, 32405, 101, 53898, 223, 94355, 103, 32405, 108, 32405, 123, 32405, 115, 53898, 235, 32405, 243, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 32405, 110, 53898, 233, 94355, 101, 32405, 123, 32405, 103, 53898, 223, 32405, 96, 53898, 223, 32405, 110, 53898, 230, 32405, 101, 94355, 116, 32405, 117, 32405, 122, 32405, 107, 32405, 243, 53898, 223, 32405, 94, 32405, 123, 94355, 103, 32405, 122, 32405, 97, 53898, 235, 32405, 108, 32405, 110, 53898, 233, 94355, 231, 32405, 101, 53898, 235, 32405, 101, 32405, 122, 32405, 108, 53898, 223, 13, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 228, 32405, 110, 53898, 233, 32405, 248, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 99, 32405, 114, 32405, 101, 53898, 223, 94355, 116, 53898, 235, 32405, 103, 32405, 115, 53898, 235, 32405, 253, 32405, 224, 32405, 245, 32405, 122, 94355, 113, 32405, 123, 32405, 113, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 330, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 3047, 94355, 227, 32405, 101, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 122, 32405, 108, 32405, 224, 32405, 255, 32405, 123, 32405, 224, 32405, 248, 32405, 123, 11, 94355, 110, 53898, 228, 32405, 243, 53898, 235, 32405, 243, 32405, 110, 53898, 223, 94355, 97, 32405, 108, 53898, 235, 32405, 243, 32405, 105, 32405, 99, 53898, 235, 32405, 100, 32405, 224, 32405, 245, 32405, 122, 94355, 248, 53898, 229, 32405, 107, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 248, 32405, 123, 32405, 113, 32405, 108, 32405, 245, 32405, 122, 330, 32405, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 1630, 1210, 94355, 227, 32405, 101, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 97, 53898, 228, 32405, 110, 32405, 103, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 229, 32405, 243, 53898, 235, 32405, 243, 32405, 94, 1630, 94355, 227, 32405, 101, 53898, 229, 32405, 99, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 13, 128009, 198, 128006, 882, 128007, 271, 32405, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 25, 220, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 17, 11, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 400, 18, 11, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 228, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 87, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 94355, 240, 32405, 243, 53898, 235, 32405, 243, 53898, 232, 32405, 243, 53898, 235, 32405, 243, 32405, 253, 32405, 123, 400, 16, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 97, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 243, 53898, 232, 32405, 224, 32405, 97, 32405, 106, 32405, 224, 32405, 99, 32405, 123, 94355, 116, 53898, 235, 32405, 101, 53898, 229, 32405, 117, 32405, 123, 32405, 97, 53898, 223, 32405, 110, 32405, 243, 53898, 223, 94355, 243, 53898, 232, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 94, 53898, 235, 32405, 108, 32405, 123, 32405, 224, 32405, 243, 53898, 235, 32405, 116, 53898, 235, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 32405, 101, 53898, 223, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 94355, 243, 53898, 233, 32405, 116, 32405, 224, 94355, 228, 32405, 106, 53898, 228, 94355, 236, 32405, 224, 32405, 97, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 32405, 101, 53898, 223, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 223, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 30, 94355, 103, 53898, 230, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 18, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 107, 32405, 101, 32405, 123, 94355, 113, 53898, 229, 32405, 108, 32405, 123, 32405, 107, 32405, 105, 53898, 223, 32405, 110, 53898, 235, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 94355, 236, 32405, 224, 32405, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
+{'input_ids': [128000, 128006, 9125, 128007, 271, 32405, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 245, 32405, 96, 32405, 123, 32405, 97, 94355, 116, 32405, 106, 32405, 116, 53898, 235, 32405, 107, 32405, 110, 32405, 101, 53898, 223, 94355, 103, 32405, 108, 32405, 123, 32405, 115, 53898, 235, 32405, 243, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 32405, 110, 53898, 233, 94355, 101, 32405, 123, 32405, 103, 53898, 223, 32405, 96, 53898, 223, 32405, 110, 53898, 230, 32405, 101, 94355, 116, 32405, 117, 32405, 122, 32405, 107, 32405, 243, 53898, 223, 32405, 94, 32405, 123, 94355, 103, 32405, 122, 32405, 97, 53898, 235, 32405, 108, 32405, 110, 53898, 233, 94355, 231, 32405, 101, 53898, 235, 32405, 101, 32405, 122, 32405, 108, 53898, 223, 13, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 228, 32405, 110, 53898, 233, 32405, 248, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 99, 32405, 114, 32405, 101, 53898, 223, 94355, 116, 53898, 235, 32405, 103, 32405, 115, 53898, 235, 32405, 253, 32405, 224, 32405, 245, 32405, 122, 94355, 113, 32405, 123, 32405, 113, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 330, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 3047, 94355, 227, 32405, 101, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 122, 32405, 108, 32405, 224, 32405, 255, 32405, 123, 32405, 224, 32405, 248, 32405, 123, 11, 94355, 110, 53898, 228, 32405, 243, 53898, 235, 32405, 243, 32405, 110, 53898, 223, 94355, 97, 32405, 108, 53898, 235, 32405, 243, 32405, 105, 32405, 99, 53898, 235, 32405, 100, 32405, 224, 32405, 245, 32405, 122, 94355, 248, 53898, 229, 32405, 107, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 248, 32405, 123, 32405, 113, 32405, 108, 32405, 245, 32405, 122, 330, 32405, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 1630, 1210, 94355, 227, 32405, 101, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 97, 53898, 228, 32405, 110, 32405, 103, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 229, 32405, 243, 53898, 235, 32405, 243, 32405, 94, 1630, 94355, 227, 32405, 101, 53898, 229, 32405, 99, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 13, 128009, 198, 128006, 882, 128007, 271, 32405, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 25, 220, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 17, 11, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 400, 18, 11, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 228, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 87, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 94355, 240, 32405, 243, 53898, 235, 32405, 243, 53898, 232, 32405, 243, 53898, 235, 32405, 243, 32405, 253, 32405, 123, 400, 16, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 97, 32405, 101, 3240 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -1005, 243, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 243, 53898, 232, 32405, 224, 32405, 97, 32405, 106, 32405, 224, 32405, 99, 32405, 123, 94355, 116, 53898, 235, 32405, 101, 53898, 229, 32405, 117, 32405, 123, 32405, 97, 53898, 223, 32405, 110, 32405, 243, 53898, 223, 94355, 243, 53898, 232, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 94, 53898, 235, 32405, 108, 32405, 123, 32405, 224, 32405, 243, 53898, 235, 32405, 116, 53898, 235, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 32405, 101, 53898, 223, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 94355, 243, 53898, 233, 32405, 116, 32405, 224, 94355, 228, 32405, 106, 53898, 228, 94355, 236, 32405, 224, 32405, 97, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 32405, 101, 53898, 223, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 223, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 30, 94355, 103, 53898, 230, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 18, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 107, 32405, 101, 32405, 123, 94355, 113, 53898, 229, 32405, 108, 32405, 123, 32405, 107, 32405, 105, 53898, 223, 32405, 110, 53898, 235, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 94355, 236, 32405, 224, 32405, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, {'input_ids': [128000, 128006, 9125, 128007, 271, 32405, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 245, 32405, 96, 32405, 123, 32405, 97, 94355, 116, 32405, 106, 32405, 116, 53898, 235, 32405, 107, 32405, 110, 32405, 101, 53898, 223, 94355, 103, 32405, 108, 32405, 123, 32405, 115, 53898, 235, 32405, 243, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 32405, 110, 53898, 233, 94355, 101, 32405, 123, 32405, 103, 53898, 223, 32405, 96, 53898, 223, 32405, 110, 53898, 230, 32405, 101, 94355, 116, 32405, 117, 32405, 122, 32405, 107, 32405, 243, 53898, 223, 32405, 94, 32405, 123, 94355, 103, 32405, 122, 32405, 97, 53898, 235, 32405, 108, 32405, 110, 53898, 233, 94355, 231, 32405, 101, 53898, 235, 32405, 101, 32405, 122, 32405, 108, 53898, 223, 13, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 228, 32405, 110, 53898, 233, 32405, 248, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 99, 32405, 114, 32405, 101, 53898, 223, 94355, 116, 53898, 235, 32405, 103, 32405, 115, 53898, 235, 32405, 253, 32405, 224, 32405, 245, 32405, 122, 94355, 113, 32405, 123, 32405, 113, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 330, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 3047, 94355, 227, 32405, 101, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 122, 32405, 108, 32405, 224, 32405, 255, 32405, 123, 32405, 224, 32405, 248, 32405, 123, 11, 94355, 110, 53898, 228, 32405, 243, 53898, 235, 32405, 243, 32405, 110, 53898, 223, 94355, 97, 32405, 108, 53898, 235, 32405, 243, 32405, 105, 32405, 99, 53898, 235, 32405, 100, 32405, 224, 32405, 245, 32405, 122, 94355, 248, 53898, 229, 32405, 107, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 248, 32405, 123, 32405, 113, 32405, 108, 32405, 245, 32405, 122, 330, 32405, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 1630, 1210, 94355, 227, 32405, 101, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 97, 53898, 228, 32405, 110, 32405, 103, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 229, 32405, 243, 53898, 235, 32405, 243, 32405, 94, 1630, 94355, 227, 32405, 101, 53898, 229, 32405, 99, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 13, 128009, 198, 128006, 882, 128007, 271, 32405, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 25, 220, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 17, 11, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 400, 18, 11, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 228, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 87, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 94355, 240, 32405, 243, 53898, 235, 32405, 243, 53898, 232, 32405, 243, 53898, 235, 32405, 243, 32405, 253, 32405, 123, 400, 16, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 97, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 243, 53898, 232, 32405, 224, 32405, 97, 32405, 106, 32405, 224, 32405, 99, 32405, 123, 94355, 116, 53898, 235, 32405, 101, 53898, 229, 32405, 117, 32405, 123, 32405, 97, 53898, 223, 32405, 110, 32405, 243, 53898, 223, 94355, 243, 53898, 232, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 94, 53898, 235, 32405, 108, 32405, 123, 32405, 224, 32405, 243, 53898, 235, 32405, 116, 53898, 235, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 32405, 101, 53898, 223, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 94355, 243, 53898, 233, 32405, 116, 32405, 224, 94355, 228, 32405, 106, 53898, 228, 94355, 236, 32405, 224, 32405, 97, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 32405, 101, 53898, 223, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 223, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 30, 94355, 103, 53898, 230, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 18, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 107, 32405, 101, 32405, 123, 94355, 113, 53898, 229, 32405, 108, 32405, 123, 32405, 107, 32405, 105, 53898, 223, 32405, 110, 53898, 235, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 94355, 236, 32405, 224, 32405, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, , -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 4001, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110,94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, , 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009]}
+ 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009]}
+1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 32405, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 245, 32405, 96, 32405, 123, 32405, 97, 94355, 116, 32405, 106, 32405, 116, 53898, 235, 32405, 107, 32405, 110, 32405, 101, 53898, 223, 94355, 103, 32405, 108, 32405, 123, 32405, 115, 53898, 235, 32405, 243, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 32405, 110, 53898, 233, 94355, 101, 32405, 123, 32405, 103, 53898, 223, 32405, 96, 53898, 223, 32405, 110, 53898, 230, 32405, 101, 94355, 116, 32405, 117, 32405, 122, 32405, 107, 32405, 243, 53898, 223, 32405, 94, 32405, 123, 94355, 103, 32405, 122, 32405, 97, 53898, 235, 32405, 108, 32405, 110, 53898, 233, 94355, 231, 32405, 101, 53898, 235, 32405, 101, 32405, 122, 32405, 108, 53898, 223, 13, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 228, 32405, 110, 53898, 233, 32405, 248, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 99, 32405, 114, 32405, 101, 53898, 223, 94355, 116, 53898, 235, 32405, 103, 32405, 115, 53898, 235, 32405, 253, 32405, 224, 32405, 245, 32405, 122, 94355, 113, 32405, 123, 32405, 113, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 330, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 3047, 94355, 227, 32405, 101, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 122, 32405, 108, 32405, 224, 32405, 255, 32405, 123, 32405, 224, 32405, 248, 32405, 123, 11, 94355, 110, 53898, 228, 32405, 243, 53898, 235, 32405, 243, 32405, 110, 53898, 223, 94355, 97, 32405, 108, 53898, 235, 32405, 243, 32405, 105, 32405, 99, 53898, 235, 32405, 100, 32405, 224, 32405, 245, 32405, 122, 94355, 248, 53898, 229, 32405, 107, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 248, 32405, 123, 32405, 113, 32405, 108, 32405, 245, 32405, 122, 330, 32405, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 1630, 1210, 94355, 227, 32405, 101, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 97, 53898, 228, 32405, 110, 32405, 103, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 229, 32405, 243, 53898, 235, 32405, 243, 32405, 94, 1630, 94355, 227, 32405, 101, 53898, 229, 32405, 99, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 13, 128009, 198, 128006, 882, 128007, 271, 32405, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 25, 220, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 17, 11, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 400, 18, 11, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 228, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 87, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 94355, 240, 32405, 243, 53898, 235, 32405, 243, 53898, 232, 32405, 243, 53898, 235, 32405, 243, 32405, 253, 32405, 123, 400, 16, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 97, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 243, 53898, 232, 32405, 224, 32405, 97, 32405, 106, 32405, 224, 32405, 99, 32405, 123, 94355, 116, 53898, 235, 32405, 101, 53898, 229, 32405, 117, 32405, 123, 32405, 97, 53898, 223, 32405, 110, 32405, 243, 53898, 223, 94355, 243, 53898, 232, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 94, 53898, 235, 32405, 108, 32405, 123, 32405, 224, 32405, 243, 53898, 235, 32405, 116, 53898, 235, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 32405, 101, 53898, 223, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 94355, 243, 53898, 233, 32405, 116, 32405, 224, 94355, 228, 32405, 106, 53898, 228, 94355, 236, 32405, 224, 32405, 97, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 32405, 101, 53898, 223, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 223, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 30, 94355, 103, 53898, 230, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 18, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 107, 32405, 101, 32405, 123, 94355, 113, 53898, 229, 32405, 108, 32405, 123, 32405, 107, 32405, 105, 53898, 223, 32405, 110, 53898, 235, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 94355, 236, 32405, 224, 32405, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, {'input_ids': [128000, 128006, 9125, 128007, 271, 32405, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 245, 32405, 96, 32405, 123, 32405, 97, 94355, 116, 32405, 106, 32405, 116, 53898, 235, 32405, 107, 32405, 110, 32405, 101, 53898, 223, 94355, 103, 32405, 108, 32405, 123, 32405, 115, 53898, 235, 32405, 243, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 32405, 110, 53898, 233, 94355, 101, 32405, 123, 32405, 103, 53898, 223, 32405, 96, 53898, 223, 32405, 110, 53898, 230, 32405, 101, 94355, 116, 32405, 117, 32405, 122, 32405, 107, 32405, 243, 53898, 223, 32405, 94, 32405, 123, 94355, 103, 32405, 122, 32405, 97, 53898, 235, 32405, 108, 32405, 110, 53898, 233, 94355, 231, 32405, 101, 53898, 235, 32405, 101, 32405, 122, 32405, 108, 53898, 223, 13, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 228, 32405, 110, 53898, 233, 32405, 248, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 99, 32405, 114, 32405, 101, 53898, 223, 94355, 116, 53898, 235, 32405, 103, 32405, 115, 53898, 235, 32405, 253, 32405, 224, 32405, 245, 32405, 122, 94355, 113, 32405, 123, 32405, 113, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 330, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 3047, 94355, 227, 32405, 101, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 122, 32405, 108, 32405, 224, 32405, 255, 32405, 123, 32405, 224, 32405, 248, 32405, 123, 11, 94355, 110, 53898, 228, 32405, 243, 53898, 235, 32405, 243, 32405, 110, 53898, 223, 94355, 97, 32405, 108, 53898, 235, 32405, 243, 32405, 105, 32405, 99, 53898, 235, 32405, 100, 32405, 224, 32405, 245, 32405, 122, 94355, 248, 53898, 229, 32405, 107, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 248, 32405, 123, 32405, 113, 32405, 108, 32405, 245, 32405, 122, 330, 32405, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 1630, 1210, 94355, 227, 32405, 101, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 97, 53898, 228, 32405, 110, 32405, 103, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 229, 32405, 243, 53898, 235, 32405, 243, 32405, 94, 1630, 94355, 227, 32405, 101, 53898, 229, 32405, 99, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 13, 128009, 198, 128006, 882, 128007, 271, 32405, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 25, 220, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 17, 11, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 400, 18, 11, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 228, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 87, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 94355, 240, 32405, 243, 53898, 235, 32405, 243, 53898, 232, 32405, 243, 53898, 235, 32405, 243, 32405, 253, 32405, 123, 400, 16, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 97, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 243, 53898, 232, 32405, 224, 32405, 97, 32405, 106, 32405, 224, 32405, 99, 32405, 123, 94355, 116, 53898, 235, 32405, 101, 53898, 229, 32405, 117, 32405, 123, 32405, 97, 53898, 223, 32405, 110, 32405, 243, 53898, 223, 94355, 243, 53898, 232, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 94, 53898, 235, 32405, 108, 32405, 123, 32405, 224, 32405, 243, 53898, 235, 32405, 116, 53898, 235, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 32405, 101, 53898, 223, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 94355, 243, 53898, 233, 32405, 116, 32405, 224, 94355, 228, 32405, 106, 53898, 228, 94355, 236, 32405, 224, 32405, 97, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 32405, 101, 53898, 223, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 223, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 30, 94355, 103, 53898, 230, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 18, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 107, 32405, 101, 32405, 123, 94355, 113, 53898, 229, 32405, 108, 32405, 123, 32405, 107, 32405, 105, 53898, 223, 32405, 110, 53898, 235, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 94355, 236, 32405, 224, 32405, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110,94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009]}
+ 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009]}
+{'input_ids': [128000, 128006, 9125, 128007, 271, 32405, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 245, 32405, 96, 32405, 123, 32405, 97, 94355, 116, 32405, 106, 32405, 116, 53898, 235, 32405, 107, 32405, 110, 32405, 101, 53898, 223, 94355, 103, 32405, 108, 32405, 123, 32405, 115, 53898, 235, 32405, 243, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 32405, 110, 53898, 233, 94355, 101, 32405, 123, 32405, 103, 53898, 223, 32405, 96, 53898, 223, 32405, 110, 53898, 230, 32405, 101, 94355, 116, 32405, 117, 32405, 122, 32405, 107, 32405, 243, 53898, 223, 32405, 94, 32405, 123, 94355, 103, 32405, 122, 32405, 97, 53898, 235, 32405, 108, 32405, 110, 53898, 233, 94355, 231, 32405, 101, 53898, 235, 32405, 101, 32405, 122, 32405, 108, 53898, 223, 13, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 53898, 222, 32405, 108, 53898, 223, 94355, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 228, 32405, 110, 53898, 233, 32405, 248, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 103, 53898, 235, 32405, 108, 32405, 97, 32405, 123, 94355, 99, 32405, 114, 32405, 101, 53898, 223, 94355, 116, 53898, 235, 32405, 103, 32405, 115, 53898, 235, 32405, 253, 32405, 224, 32405, 245, 32405, 122, 94355, 113, 32405, 123, 32405, 113, 32405, 108, 32405, 123, 32405, 224, 32405, 248, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 330, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 3047, 94355, 227, 32405, 101, 32405, 123, 94355, 103, 53898, 235, 32405, 108, 32405, 122, 32405, 108, 32405, 224, 32405, 255, 32405, 123, 32405, 224, 32405, 248, 32405, 123, 11, 94355, 110, 53898, 228, 32405, 243, 53898, 235, 32405, 243, 32405, 110, 53898, 223, 94355, 97, 32405, 108, 53898, 235, 32405, 243, 32405, 105, 32405, 99, 53898, 235, 32405, 100, 32405, 224, 32405, 245, 32405, 122, 94355, 248, 53898, 229, 32405, 107, 32405, 122, 32405, 110, 32405, 123, 13, 94355, 248, 32405, 123, 32405, 113, 32405, 108, 32405, 245, 32405, 122, 330, 32405, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 1630, 1210, 94355, 227, 32405, 101, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 122, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 97, 53898, 228, 32405, 110, 32405, 103, 32405, 122, 32405, 110, 32405, 123, 11, 94355, 229, 32405, 243, 53898, 235, 32405, 243, 32405, 94, 1630, 94355, 227, 32405, 101, 53898, 229, 32405, 99, 32405, 123, 94355, 97, 53898, 223, 32405, 99, 32405, 123, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 13, 128009, 198, 128006, 882, 128007, 271, 32405, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 25, 220, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 17, 11, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 400, 18, 11, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 228, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 400, 87, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 116, 53898, 235, 94355, 100, 32405, 108, 94355, 240, 32405, 243, 53898, 235, 32405, 243, 53898, 232, 32405, 243, 53898, 235, 32405, 243, 32405, 253, 32405, 123, 400, 16, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 97, 32405, 101, 32405, 243, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 243, 53898, 232, 32405, 224, 32405, 97, 32405, 106, 32405, 224, 32405, 99, 32405, 123, 94355, 116, 53898, 235, 32405, 101, 53898, 229, 32405, 117, 32405, 123, 32405, 97, 53898, 223, 32405, 110, 32405, 243, 53898, 223, 94355, 243, 53898, 232, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 94, 53898, 235, 32405, 108, 32405, 123, 32405, 224, 32405, 243, 53898, 235, 32405, 116, 53898, 235, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 53898, 223, 11, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 53898, 223, 94355, 106, 32405, 108, 32405, 123, 32405, 107, 53898, 223, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 32405, 101, 53898, 223, 94355, 228, 32405, 108, 53898, 235, 32405, 94, 32405, 108, 53898, 235, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 13, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 94355, 243, 53898, 233, 32405, 116, 32405, 224, 94355, 228, 32405, 106, 53898, 228, 94355, 236, 32405, 224, 32405, 97, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 32405, 101, 53898, 223, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 223, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 30, 94355, 103, 53898, 230, 94355, 103, 53898, 235, 32405, 108, 32405, 114, 53898, 235, 32405, 101, 32405, 243, 53898, 223, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 18, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 97, 53898, 228, 32405, 110, 32405, 123, 32405, 107, 32405, 101, 32405, 123, 94355, 113, 53898, 229, 32405, 108, 32405, 123, 32405, 107, 32405, 105, 53898, 223, 32405, 110, 53898, 235, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 94355, 236, 32405, 224, 32405, 97, 30, 128009, 198, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 128006, 78191, 128007, 271, 32405, 99, 32405, 114, 32405, 110, 32405, 113, 32405, 122, 32405, 108, 53898, 222, 32405, 245, 32405, 122, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 25, 94355, 106, 53898, 224, 32405, 94, 53898, 223, 94355, 243, 32405, 122, 32405, 103, 53898, 223, 32405, 248, 32405, 123, 32405, 101, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 18, 353, 400, 17, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 238, 32405, 116, 53898, 235, 90464, 32405, 94, 53898, 235, 94355, 253, 53898, 222, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 18, 284, 400, 21, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 243, 53898, 229, 32405, 104, 53898, 235, 94355, 110, 32405, 122, 32405, 253, 53898, 235, 90464, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 87, 284, 400, 17, 87, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 236, 32405, 116, 53898, 235, 32405, 103, 53898, 235, 32405, 108, 53898, 228, 32405, 116, 53898, 235, 32405, 116, 53898, 233, 32405, 110, 94355, 100, 32405, 108, 220, 17, 353, 400, 16, 284, 400, 17, 13, 94355, 227, 32405, 101, 53898, 235, 32405, 101, 32405, 123, 94355, 103, 32405, 122, 32405, 101, 53898, 222, 32405, 107, 32405, 122, 32405, 110, 94355, 106, 53898, 232, 32405, 97, 53898, 235, 32405, 97, 32405, 224, 94355, 100, 32405, 108, 400, 21, 489, 400, 21, 489, 400, 17, 87, 489, 400, 17, 284, 400, 975, 489, 400, 17, 87, 13, 94355, 114, 32405, 122, 32405, 224, 32405, 94, 53898, 222, 94355, 229, 32405, 108, 32405, 113, 53898, 230, 12, 32405, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 94355, 105, 32405, 123, 32405, 110, 53898, 235, 32405, 110, 53898, 223, 32405, 97, 53898, 233, 94355, 248, 53898, 228, 32405, 110, 53898, 235, 32405, 110, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 53898, 229, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 400, 508, 482, 1746, 975, 489, 400, 17, 87, 8, 284, 400, 508, 482, 400, 975, 482, 400, 17, 87, 284, 400, 21, 482, 400, 17, 87, 13, 94355, 228, 32405, 106, 53898, 228, 94355, 97, 32405, 123, 32405, 108, 32405, 123, 32405, 245, 32405, 123, 94355, 103, 53898, 232, 32405, 224, 32405, 99, 32405, 123, 32405, 101, 94355, 106, 32405, 122, 32405, 108, 53898, 235, 32405, 103, 53898, 223, 220, 18, 94355, 94, 32405, 122, 32405, 110, 32405, 108, 53898, 235, 32405, 110, 53898, 223, 94355, 227, 32405, 101, 32405, 123, 94355, 106, 32405, 122, 32405, 243, 53898, 223, 94355, 227, 32405, 224, 32405, 99, 32405, 123, 32405, 224, 32405, 248, 32405, 105, 32405, 94, 32405, 123, 32405, 224, 32405, 99, 32405, 123, 11, 94355, 243, 32405, 122, 32405, 105, 32405, 253, 53898, 235, 32405, 253, 32405, 123, 94355, 106, 32405, 101, 32405, 224, 94355, 229, 32405, 110, 32405, 122, 94355, 113, 53898, 235, 32405, 108, 32405, 122, 32405, 107, 32405, 113, 32405, 248, 53898, 235, 32405, 248, 53898, 223, 25, 400, 21, 482, 400, 17, 87, 284, 400, 18, 13, 94355, 116, 32405, 108, 32405, 111, 53898, 222, 32405, 243, 53898, 225, 32405, 97, 32405, 224, 94355, 248, 53898, 229, 32405, 116, 53898, 235, 32405, 97, 53898, 229, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 400, 17, 87, 284, 400, 18, 13, 94355, 108, 53898, 228, 32405, 224, 32405, 94, 53898, 223, 94355, 113, 53898, 230, 32405, 103, 53898, 223, 32405, 110, 32405, 122, 400, 17, 94355, 99, 53898, 235, 32405, 113, 32405, 122, 32405, 108, 32405, 122, 94355, 113, 32405, 123, 32405, 255, 32405, 250, 32405, 123, 32405, 224, 32405, 248, 32405, 94, 32405, 224, 11, 94355, 106, 32405, 101, 32405, 243, 53898, 223, 94355, 110, 32405, 255, 32405, 123, 32405, 116, 53898, 235, 32405, 97, 53898, 223, 32405, 224, 32405, 99, 32405, 123, 25, 865, 284, 400, 16, 13, 1135, 13, 865, 94355, 113, 32405, 123, 32405, 110, 53898, 223, 32405, 113, 400, 16, 13, 1135, 13, 94355, 116, 32405, 106, 32405, 122, 32405, 100, 32405, 122, 32405, 101, 32405, 224, 220, 16, 13, 20, 13, 128009]}
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/utils/import_utils.py:560: FutureWarning: `is_torch_tpu_available` is deprecated and will be removed in 4.41.0. Please use the `is_torch_xla_available` instead.
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+/home/iitm_admin/llmteam/mid-align/src/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
+  warnings.warn(
+[INFO|trainer.py:658] 2025-06-26 22:15:41,244 >> Using auto half precision backend
+[2025-06-26 22:15:41,524] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed info: version=0.17.1, git-hash=unknown, git-branch=unknown
+[2025-06-26 22:15:41,524] [INFO] [config.py:655:__init__] Config mesh_device None world_size = 4
+[2025-06-26 22:15:45,166] [INFO] [engine.py:1325:_configure_distributed_model] ********** distributed groups summary **********
+	 self.dp_world_size=4
+	 self.mp_world_size=1
+	 self.seq_dp_world_size=4
+	 self.sequence_parallel_size=1
+***********************************************
+[2025-06-26 22:15:46,246] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Detected CUDA files, patching ldflags
+Emitting ninja build file /home/iitm_admin/.cache/torch_extensions/py39_cu126/cpu_adam/build.ninja...
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/utils/cpp_extension.py:2356: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
+If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
+  warnings.warn(
+Building extension module cpu_adam...
+Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
+ninja: no work to do.
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 2.6924383640289307 seconds
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 2.6254212856292725 seconds
+Adam Optimizer #0 is created with AVX512 arithmetic capability.
+Config: alpha=0.000500, betas=(0.900000, 0.999000), weight_decay=0.000000, adam_w=1
+[2025-06-26 22:15:50,318] [INFO] [logging.py:107:log_dist] [Rank 0] Using DeepSpeed Optimizer param name adam as basic optimizer
+[2025-06-26 22:15:50,319] [INFO] [logging.py:107:log_dist] [Rank 0] Removing param_group that has no 'params' in the basic Optimizer
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+Installed CUDA version 12.0 does not match the version torch was compiled with 12.6 but since the APIs are compatible, accepting this combination
+Using /home/iitm_admin/.cache/torch_extensions/py39_cu126 as PyTorch extensions root...
+[2025-06-26 22:15:50,389] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Basic Optimizer = DeepSpeedCPUAdam
+[2025-06-26 22:15:50,390] [INFO] [utils.py:59:is_zero_supported_optimizer] Checking ZeRO support for optimizer=DeepSpeedCPUAdam type=<class 'deepspeed.ops.adam.cpu_adam.DeepSpeedCPUAdam'>
+[2025-06-26 22:15:50,390] [INFO] [logging.py:107:log_dist] [Rank 0] Creating torch.bfloat16 ZeRO stage 1 optimizer
+[2025-06-26 22:15:50,390] [INFO] [stage_1_and_2.py:151:__init__] Reduce bucket size 200000000
+[2025-06-26 22:15:50,390] [INFO] [stage_1_and_2.py:152:__init__] Allgather bucket size 200000000
+[2025-06-26 22:15:50,390] [INFO] [stage_1_and_2.py:153:__init__] CPU Offload: True
+[2025-06-26 22:15:50,390] [INFO] [stage_1_and_2.py:154:__init__] Round robin gradient partitioning: False
+Detected CUDA files, patching ldflags
+Emitting ninja build file /home/iitm_admin/.cache/torch_extensions/py39_cu126/cpu_adam/build.ninja...
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/utils/cpp_extension.py:2356: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
+If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
+  warnings.warn(
+Building extension module cpu_adam...
+Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
+ninja: no work to do.
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 2.794114828109741 seconds
+Loading extension module cpu_adam...
+Time to load cpu_adam op: 2.826972007751465 seconds
+[2025-06-26 22:15:50,795] [INFO] [utils.py:781:see_memory_usage] Before initializing optimizer states
+[2025-06-26 22:15:50,796] [INFO] [utils.py:782:see_memory_usage] MA 15.0 GB         Max_MA 15.0 GB         CA 15.16 GB         Max_CA 15 GB 
+[2025-06-26 22:15:50,796] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 77.29 GB, percent = 3.8%
+[2025-06-26 22:15:50,994] [INFO] [utils.py:781:see_memory_usage] After initializing optimizer states
+[2025-06-26 22:15:50,995] [INFO] [utils.py:782:see_memory_usage] MA 15.0 GB         Max_MA 15.0 GB         CA 15.16 GB         Max_CA 15 GB 
+[2025-06-26 22:15:50,995] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 77.38 GB, percent = 3.8%
+[2025-06-26 22:15:50,995] [INFO] [stage_1_and_2.py:573:__init__] optimizer state initialized
+[2025-06-26 22:15:51,142] [INFO] [utils.py:781:see_memory_usage] After initializing ZeRO optimizer
+[2025-06-26 22:15:51,143] [INFO] [utils.py:782:see_memory_usage] MA 15.0 GB         Max_MA 15.0 GB         CA 15.16 GB         Max_CA 15 GB 
+[2025-06-26 22:15:51,143] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 77.42 GB, percent = 3.8%
+[2025-06-26 22:15:51,145] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Final Optimizer = DeepSpeedZeroOptimizer
+[2025-06-26 22:15:51,146] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed using client callable to create LR scheduler
+[2025-06-26 22:15:51,146] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed LR Scheduler = <torch.optim.lr_scheduler.LambdaLR object at 0x7701f451a7c0>
+[2025-06-26 22:15:51,146] [INFO] [logging.py:107:log_dist] [Rank 0] step=0, skipped=0, lr=[0.0], mom=[[0.9, 0.999]]
+[2025-06-26 22:15:51,151] [INFO] [logging.py:107:log_dist] [Rank 0] [TorchCheckpointEngine] Initialized with serialization = True
+[2025-06-26 22:15:51,151] [INFO] [config.py:921:print] DeepSpeedEngine configuration:
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   activation_checkpointing_config  {
+    "partition_activations": false, 
+    "contiguous_memory_optimization": false, 
+    "cpu_checkpointing": false, 
+    "number_checkpoints": null, 
+    "synchronize_checkpoint_boundary": false, 
+    "profile": false
+}
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'intra_op_parallelism': 1, 'single_submit': False, 'overlap_events': True, 'use_gds': False}
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   amp_enabled .................. False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   amp_params ................... False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   autotuning_config ............ {
+    "enabled": false, 
+    "start_step": null, 
+    "end_step": null, 
+    "metric_path": null, 
+    "arg_mappings": null, 
+    "metric": "throughput", 
+    "model_info": null, 
+    "results_dir": "autotuning_results", 
+    "exps_dir": "autotuning_exps", 
+    "overwrite": true, 
+    "fast": true, 
+    "start_profile_step": 3, 
+    "end_profile_step": 5, 
+    "tuner_type": "gridsearch", 
+    "tuner_early_stopping": 5, 
+    "tuner_num_trials": 50, 
+    "model_info_path": null, 
+    "mp_size": 1, 
+    "max_train_batch_size": null, 
+    "min_train_batch_size": 1, 
+    "max_train_micro_batch_size_per_gpu": 1.024000e+03, 
+    "min_train_micro_batch_size_per_gpu": 1, 
+    "num_tuning_micro_batch_sizes": 3
+}
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   bfloat16_config .............. enabled=True immediate_grad_update=False check_grad_overflow=False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   checkpoint_config ............ {'tag_validation': 'WARN', 'checkpoint_serialization': True, 'writer': None}
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   checkpoint_parallel_write_pipeline  False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   checkpoint_tag_validation_enabled  True
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   checkpoint_tag_validation_fail  False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   comms_config ................. <deepspeed.comm.config.DeepSpeedCommsConfig object at 0x7701f4ab7e20>
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   communication_data_type ...... None
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   compile_config ............... deepcompile=False free_activation=False offload_activation=False offload_opt_states=False double_buffer=True symmetric_memory=False debug_log=False offload_parameters=False sync_before_reduce=False sync_after_reduce=False sync_before_allgather=False sync_after_allgather=False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}}
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   curriculum_enabled_legacy .... False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   curriculum_params_legacy ..... False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'pin_memory': False, 'curriculum_learning': {'enabled': False}, 'dynamic_batching': {'enabled': False, 'lr_scaling_method': 'linear', 'min_batch_size': 1, 'max_batch_size': None, 'sequence_picking_order': 'dataloader', 'verbose': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}}
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   data_efficiency_enabled ...... False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   dataloader_drop_last ......... False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   disable_allgather ............ False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   dump_state ................... False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   eigenvalue_enabled ........... False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   eigenvalue_gas_boundary_resolution  1
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   eigenvalue_layer_name ........ bert.encoder.layer
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   eigenvalue_layer_num ......... 0
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   eigenvalue_max_iter .......... 100
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   eigenvalue_stability ......... 1e-06
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   eigenvalue_tol ............... 0.01
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   eigenvalue_verbose ........... False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   elasticity_enabled ........... False
+[2025-06-26 22:15:51,152] [INFO] [config.py:925:print]   float16_config ............... enabled=False auto_cast=False loss_scale=0.0 initial_scale_power=16 loss_scale_window=1000 hysteresis=2 consecutive_hysteresis=False min_loss_scale=1 fp16_master_weights_and_grads=False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   flops_profiler_config ........ {
+    "enabled": false, 
+    "recompute_fwd_factor": 0.0, 
+    "profile_step": 1, 
+    "module_depth": -1, 
+    "top_modules": 1, 
+    "detailed": true, 
+    "output_file": null
+}
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   global_rank .................. 0
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   grad_accum_dtype ............. None
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   gradient_accumulation_steps .. 1
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   gradient_clipping ............ 1.0
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   gradient_predivide_factor .... 1.0
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   graph_harvesting ............. False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   load_universal_checkpoint .... False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   memory_breakdown ............. False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   mics_hierarchial_params_gather  False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   mics_shard_size .............. -1
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') comet=CometConfig(enabled=False, samples_log_interval=100, project=None, workspace=None, api_key=None, experiment_name=None, experiment_key=None, online=None, mode=None) wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName')
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   nebula_config ................ {
+    "enabled": false, 
+    "persistent_storage_path": null, 
+    "persistent_time_interval": 100, 
+    "num_of_version_in_retention": 2, 
+    "enable_nebula_load": true, 
+    "load_path": null
+}
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   optimizer_legacy_fusion ...... False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   optimizer_name ............... adam
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   optimizer_params ............. {'lr': 0.0005, 'betas': [0.9, 0.999], 'eps': 1e-08, 'weight_decay': 0.0}
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0, 'pipe_partitioned': True, 'grad_partitioned': True}
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   pld_enabled .................. False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   pld_params ................... False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   prescale_gradients ........... False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   scheduler_name ............... None
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   scheduler_params ............. None
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   seq_parallel_communication_data_type  torch.float32
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   sparse_attention ............. None
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   sparse_gradients_enabled ..... False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   steps_per_print .............. inf
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   tensor_parallel_config ....... dtype=torch.float16 autotp_size=0 tp_overlap_comm=False tensor_parallel=TPConfig(tp_size=1, tp_grain_size=1, mpu=None, tp_group=None) injection_policy_tuple=None keep_module_on_host=False replace_with_kernel_inject=False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   timers_config ................ enabled=True synchronized=True
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   train_batch_size ............. 100
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   train_micro_batch_size_per_gpu  25
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   use_data_before_expert_parallel_  False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   use_node_local_storage ....... False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   wall_clock_breakdown ......... False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   weight_quantization_config ... None
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   world_size ................... 4
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   zero_allow_untested_optimizer  False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   zero_config .................. stage=1 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=200000000 use_multi_rank_bucket_allreduce=True allgather_partitions=True allgather_bucket_size=200000000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=None offload_optimizer=DeepSpeedZeroOffloadOptimizerConfig(device='cpu', nvme_path=None, buffer_count=4, pin_memory=True, pipeline_read=False, pipeline_write=False, fast_init=False, ratio=1.0) sub_group_size=1000000000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50000000 param_persistence_threshold=100000 model_persistence_threshold=9223372036854775807 max_live_parameters=1000000000 max_reuse_distance=1000000000 gather_16bit_weights_on_model_save=False module_granularity_threshold=0 use_all_reduce_for_fetch_params=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False zero_hpz_partition_size=1 zero_quantized_weights=False zero_quantized_nontrainable_weights=False zero_quantized_gradients=False zeropp_loco_param=None mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=True pipeline_loading_checkpoint=False override_module_apply=True log_trace_cache_warnings=False
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   zero_enabled ................. True
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   zero_force_ds_cpu_optimizer .. True
+[2025-06-26 22:15:51,153] [INFO] [config.py:925:print]   zero_optimization_stage ...... 1
+[2025-06-26 22:15:51,153] [INFO] [config.py:911:print_user_config]   json = {
+    "optimizer": {
+        "type": "Adam", 
+        "params": {
+            "lr": 0.0005, 
+            "betas": [0.9, 0.999], 
+            "eps": 1e-08, 
+            "weight_decay": 0.0
+        }
+    }, 
+    "bf16": {
+        "enabled": true
+    }, 
+    "fp16": {
+        "enabled": false, 
+        "loss_scale": 0, 
+        "loss_scale_window": 1000, 
+        "initial_scale_power": 16, 
+        "hysteresis": 2, 
+        "min_loss_scale": 1
+    }, 
+    "zero_optimization": {
+        "stage": 1, 
+        "offload_optimizer": {
+            "device": "cpu", 
+            "pin_memory": true
+        }, 
+        "allgather_partitions": true, 
+        "allgather_bucket_size": 2.000000e+08, 
+        "overlap_comm": true, 
+        "reduce_scatter": true, 
+        "reduce_bucket_size": 2.000000e+08, 
+        "contiguous_gradients": true
+    }, 
+    "gradient_accumulation_steps": 1, 
+    "gradient_clipping": 1.0, 
+    "steps_per_print": inf, 
+    "train_batch_size": 100, 
+    "train_micro_batch_size_per_gpu": 25, 
+    "wall_clock_breakdown": false
+}
+[INFO|trainer.py:2145] 2025-06-26 22:15:51,155 >> ***** Running training *****
+[INFO|trainer.py:2146] 2025-06-26 22:15:51,155 >>   Num examples = 15,000
+[INFO|trainer.py:2147] 2025-06-26 22:15:51,155 >>   Num Epochs = 5
+[INFO|trainer.py:2148] 2025-06-26 22:15:51,155 >>   Instantaneous batch size per device = 25
+[INFO|trainer.py:2151] 2025-06-26 22:15:51,155 >>   Total train batch size (w. parallel, distributed & accumulation) = 100
+[INFO|trainer.py:2152] 2025-06-26 22:15:51,155 >>   Gradient Accumulation steps = 1
+[INFO|trainer.py:2153] 2025-06-26 22:15:51,155 >>   Total optimization steps = 750
+[INFO|trainer.py:2154] 2025-06-26 22:15:51,159 >>   Number of trainable parameters = 20,971,520
+[INFO|integration_utils.py:807] 2025-06-26 22:15:51,162 >> Automatic Weights & Biases logging enabled, to disable set os.environ["WANDB_DISABLED"] = "true"
+wandb: WARNING The `run_name` is currently set to the same value as `TrainingArguments.output_dir`. If this was not intended, please specify a different run name by setting the `TrainingArguments.run_name` parameter.
+wandb: Currently logged in as: sidharthpulipaka (indic-encoder) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
+wandb: Tracking run with wandb version 0.20.1
+wandb: Run data is saved locally in /home/iitm_admin/llmteam/mid-align/wandb/run-20250626_221551-wms07pcr
+wandb: Run `wandb offline` to turn off syncing.
+wandb: Syncing run ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/
+wandb: ⭐️ View project at https://wandb.ai/indic-encoder/midalign
+wandb: 🚀 View run at https://wandb.ai/indic-encoder/midalign/runs/wms07pcr
+
  0%|          | 0/750 [00:00<?, ?it/s][INFO|trainer.py:3831] 2025-06-26 22:15:53,180 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-26 22:15:53,180 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-26 22:15:53,180 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:01<00:07,  1.10it/s]
+
 30%|███       | 3/10 [00:03<00:09,  1.30s/it]
+
 40%|████      | 4/10 [00:05<00:09,  1.50s/it]
+
 50%|█████     | 5/10 [00:07<00:08,  1.62s/it]
+
 60%|██████    | 6/10 [00:09<00:06,  1.69s/it]wandb: WARNING Fatal error while uploading data. Some run data will not be synced, but it will still be written to disk. Use `wandb sync` at the end of the run to try uploading.
+
+
 70%|███████   | 7/10 [00:11<00:05,  1.73s/it]
+
 80%|████████  | 8/10 [00:12<00:03,  1.76s/it]
+
 90%|█████████ | 9/10 [00:14<00:01,  1.79s/it]
+
100%|██████████| 10/10 [00:18<00:00,  2.41s/it]
+
                                               

                                       
{'eval_loss': 0.22844204306602478, 'eval_accuracy': 0.28797459697117733, 'eval_runtime': 34.2122, 'eval_samples_per_second': 29.229, 'eval_steps_per_second': 0.292, 'epoch': 0}
+
+
100%|██████████| 10/10 [00:24<00:00,  2.41s/it]
  0%|          | 0/750 [00:34<?, ?it/s]
+
                                               
  0%|          | 1/750 [00:49<10:16:25, 49.38s/it]
                                                  
{'loss': 0.2378, 'grad_norm': 0.42804375290870667, 'learning_rate': 2.173913043478261e-05, 'epoch': 0.01}
+
  0%|          | 1/750 [00:49<10:16:25, 49.38s/it]
  0%|          | 2/750 [00:55<4:58:50, 23.97s/it] 
                                                 
{'loss': 0.2256, 'grad_norm': 0.402502179145813, 'learning_rate': 4.347826086956522e-05, 'epoch': 0.01}
+
  0%|          | 2/750 [00:55<4:58:50, 23.97s/it]
  0%|          | 3/750 [01:01<3:16:45, 15.80s/it]
                                                 
{'loss': 0.2305, 'grad_norm': 0.48126518726348877, 'learning_rate': 6.521739130434782e-05, 'epoch': 0.02}
+
  0%|          | 3/750 [01:01<3:16:45, 15.80s/it]
  1%|          | 4/750 [01:07<2:28:53, 11.98s/it]
                                                 
{'loss': 0.2485, 'grad_norm': 0.47943294048309326, 'learning_rate': 8.695652173913044e-05, 'epoch': 0.03}
+
  1%|          | 4/750 [01:07<2:28:53, 11.98s/it]
  1%|          | 5/750 [01:13<2:02:59,  9.91s/it]
                                                 
{'loss': 0.2071, 'grad_norm': 0.24165761470794678, 'learning_rate': 0.00010869565217391305, 'epoch': 0.03}
+
  1%|          | 5/750 [01:13<2:02:59,  9.91s/it]
  1%|          | 6/750 [01:20<1:46:56,  8.62s/it]
                                                 
{'loss': 0.2046, 'grad_norm': 0.13678190112113953, 'learning_rate': 0.00013043478260869564, 'epoch': 0.04}
+
  1%|          | 6/750 [01:20<1:46:56,  8.62s/it]
  1%|          | 7/750 [01:26<1:37:08,  7.84s/it]
                                                 
{'loss': 0.1932, 'grad_norm': 0.12917397916316986, 'learning_rate': 0.00015217391304347827, 'epoch': 0.05}
+
  1%|          | 7/750 [01:26<1:37:08,  7.84s/it]
  1%|          | 8/750 [01:32<1:30:17,  7.30s/it]
                                                 
{'loss': 0.1828, 'grad_norm': 0.15538276731967926, 'learning_rate': 0.00017391304347826088, 'epoch': 0.05}
+
  1%|          | 8/750 [01:32<1:30:17,  7.30s/it]
  1%|          | 9/750 [01:38<1:25:54,  6.96s/it]
                                                 
{'loss': 0.1833, 'grad_norm': 0.15884819626808167, 'learning_rate': 0.0001956521739130435, 'epoch': 0.06}
+
  1%|          | 9/750 [01:38<1:25:54,  6.96s/it]
  1%|▏         | 10/750 [01:44<1:23:10,  6.74s/it]
                                                  
{'loss': 0.1874, 'grad_norm': 0.0892435759305954, 'learning_rate': 0.0002173913043478261, 'epoch': 0.07}
+
  1%|▏         | 10/750 [01:44<1:23:10,  6.74s/it]
  1%|▏         | 11/750 [01:51<1:21:12,  6.59s/it]
                                                  
{'loss': 0.1761, 'grad_norm': 0.09326554089784622, 'learning_rate': 0.0002391304347826087, 'epoch': 0.07}
+
  1%|▏         | 11/750 [01:51<1:21:12,  6.59s/it]
  2%|▏         | 12/750 [01:57<1:19:56,  6.50s/it]
                                                  
{'loss': 0.1773, 'grad_norm': 0.07735300809144974, 'learning_rate': 0.0002608695652173913, 'epoch': 0.08}
+
  2%|▏         | 12/750 [01:57<1:19:56,  6.50s/it]
  2%|▏         | 13/750 [02:03<1:18:43,  6.41s/it]
                                                  
{'loss': 0.1925, 'grad_norm': 0.06787180155515671, 'learning_rate': 0.0002826086956521739, 'epoch': 0.09}
+
  2%|▏         | 13/750 [02:03<1:18:43,  6.41s/it]
  2%|▏         | 14/750 [02:09<1:17:49,  6.35s/it]
                                                  
{'loss': 0.1628, 'grad_norm': 0.06105981767177582, 'learning_rate': 0.00030434782608695655, 'epoch': 0.09}
+
  2%|▏         | 14/750 [02:09<1:17:49,  6.35s/it]
  2%|▏         | 15/750 [02:16<1:17:14,  6.31s/it]
                                                  
{'loss': 0.1727, 'grad_norm': 0.06381712853908539, 'learning_rate': 0.00032608695652173916, 'epoch': 0.1}
+
  2%|▏         | 15/750 [02:16<1:17:14,  6.31s/it]
  2%|▏         | 16/750 [02:22<1:16:58,  6.29s/it]
                                                  
{'loss': 0.1759, 'grad_norm': 0.05812380090355873, 'learning_rate': 0.00034782608695652176, 'epoch': 0.11}
+
  2%|▏         | 16/750 [02:22<1:16:58,  6.29s/it]
  2%|▏         | 17/750 [02:28<1:17:04,  6.31s/it]
                                                  
{'loss': 0.1843, 'grad_norm': 0.0629408210515976, 'learning_rate': 0.0003695652173913043, 'epoch': 0.11}
+
  2%|▏         | 17/750 [02:28<1:17:04,  6.31s/it]
  2%|▏         | 18/750 [02:35<1:17:03,  6.32s/it]
                                                  
{'loss': 0.1709, 'grad_norm': 0.06249479576945305, 'learning_rate': 0.000391304347826087, 'epoch': 0.12}
+
  2%|▏         | 18/750 [02:35<1:17:03,  6.32s/it]
  3%|▎         | 19/750 [02:41<1:16:41,  6.29s/it]
                                                  
{'loss': 0.1625, 'grad_norm': 0.05880042538046837, 'learning_rate': 0.0004130434782608696, 'epoch': 0.13}
+
  3%|▎         | 19/750 [02:41<1:16:41,  6.29s/it]
  3%|▎         | 20/750 [02:47<1:16:12,  6.26s/it]
                                                  
{'loss': 0.182, 'grad_norm': 0.06687948852777481, 'learning_rate': 0.0004347826086956522, 'epoch': 0.13}
+
  3%|▎         | 20/750 [02:47<1:16:12,  6.26s/it]
  3%|▎         | 21/750 [02:53<1:15:45,  6.24s/it]
                                                  
{'loss': 0.1702, 'grad_norm': 0.06032991036772728, 'learning_rate': 0.00045652173913043474, 'epoch': 0.14}
+
  3%|▎         | 21/750 [02:53<1:15:45,  6.24s/it]
  3%|▎         | 22/750 [02:59<1:15:12,  6.20s/it]
                                                  
{'loss': 0.1658, 'grad_norm': 0.060130324214696884, 'learning_rate': 0.0004782608695652174, 'epoch': 0.15}
+
  3%|▎         | 22/750 [02:59<1:15:12,  6.20s/it]
  3%|▎         | 23/750 [03:05<1:14:55,  6.18s/it]
                                                  
{'loss': 0.1698, 'grad_norm': 0.06124340742826462, 'learning_rate': 0.0005, 'epoch': 0.15}
+
  3%|▎         | 23/750 [03:05<1:14:55,  6.18s/it]
  3%|▎         | 24/750 [03:12<1:15:05,  6.21s/it]
                                                  
{'loss': 0.1618, 'grad_norm': 0.06329341977834702, 'learning_rate': 0.0004894725051862804, 'epoch': 0.16}
+
  3%|▎         | 24/750 [03:12<1:15:05,  6.21s/it]
  3%|▎         | 25/750 [03:18<1:14:56,  6.20s/it]
                                                  
{'loss': 0.1745, 'grad_norm': 0.06526025384664536, 'learning_rate': 0.00047958315233127195, 'epoch': 0.17}
+
  3%|▎         | 25/750 [03:18<1:14:56,  6.20s/it]
  3%|▎         | 26/750 [03:24<1:14:50,  6.20s/it]
                                                  
{'loss': 0.1795, 'grad_norm': 0.06823654472827911, 'learning_rate': 0.0004702699715629801, 'epoch': 0.17}
+
  3%|▎         | 26/750 [03:24<1:14:50,  6.20s/it]
  4%|▎         | 27/750 [03:30<1:14:22,  6.17s/it]
                                                  
{'loss': 0.1679, 'grad_norm': 0.07006911933422089, 'learning_rate': 0.00046147910349544864, 'epoch': 0.18}
+
  4%|▎         | 27/750 [03:30<1:14:22,  6.17s/it]
  4%|▎         | 28/750 [03:36<1:14:17,  6.17s/it]
                                                  
{'loss': 0.1861, 'grad_norm': 0.06462811678647995, 'learning_rate': 0.00045316348358748285, 'epoch': 0.19}
+
  4%|▎         | 28/750 [03:36<1:14:17,  6.17s/it]
  4%|▍         | 29/750 [03:43<1:14:11,  6.17s/it]
                                                  
{'loss': 0.1701, 'grad_norm': 0.06981588155031204, 'learning_rate': 0.00044528177828086066, 'epoch': 0.19}
+
  4%|▍         | 29/750 [03:43<1:14:11,  6.17s/it]
  4%|▍         | 30/750 [03:49<1:14:08,  6.18s/it]
                                                  
{'loss': 0.1556, 'grad_norm': 0.06187241151928902, 'learning_rate': 0.0004377975178854565, 'epoch': 0.2}
+
  4%|▍         | 30/750 [03:49<1:14:08,  6.18s/it]
  4%|▍         | 31/750 [03:55<1:14:05,  6.18s/it]
                                                  
{'loss': 0.1657, 'grad_norm': 0.05558977648615837, 'learning_rate': 0.0004306783846070544, 'epoch': 0.21}
+
  4%|▍         | 31/750 [03:55<1:14:05,  6.18s/it]
  4%|▍         | 32/750 [04:01<1:13:58,  6.18s/it]
                                                  
{'loss': 0.1618, 'grad_norm': 0.05787397548556328, 'learning_rate': 0.00042389562394532925, 'epoch': 0.21}
+
  4%|▍         | 32/750 [04:01<1:13:58,  6.18s/it]
  4%|▍         | 33/750 [04:07<1:13:36,  6.16s/it]
                                                  
{'loss': 0.1709, 'grad_norm': 0.06566811352968216, 'learning_rate': 0.00041742355496836097, 'epoch': 0.22}
+
  4%|▍         | 33/750 [04:07<1:13:36,  6.16s/it]
  5%|▍         | 34/750 [04:13<1:13:22,  6.15s/it]
                                                  
{'loss': 0.1774, 'grad_norm': 0.05993001163005829, 'learning_rate': 0.00041123916041498716, 'epoch': 0.23}
+
  5%|▍         | 34/750 [04:13<1:13:22,  6.15s/it]
  5%|▍         | 35/750 [04:19<1:13:06,  6.13s/it]
                                                  
{'loss': 0.1471, 'grad_norm': 0.05561774969100952, 'learning_rate': 0.00040532174168888874, 'epoch': 0.23}
+
  5%|▍         | 35/750 [04:19<1:13:06,  6.13s/it]
  5%|▍         | 36/750 [04:26<1:14:04,  6.23s/it]
                                                  
{'loss': 0.1734, 'grad_norm': 0.05735047161579132, 'learning_rate': 0.00039965262694272664, 'epoch': 0.24}
+
  5%|▍         | 36/750 [04:26<1:14:04,  6.23s/it]
  5%|▍         | 37/750 [04:32<1:13:45,  6.21s/it]
                                                  
{'loss': 0.1641, 'grad_norm': 0.057827427983284, 'learning_rate': 0.00039421492285986, 'epoch': 0.25}
+
  5%|▍         | 37/750 [04:32<1:13:45,  6.21s/it]
  5%|▌         | 38/750 [04:38<1:13:32,  6.20s/it]
                                                  
{'loss': 0.1836, 'grad_norm': 0.05822821334004402, 'learning_rate': 0.00038899330260774953, 'epoch': 0.25}
+
  5%|▌         | 38/750 [04:38<1:13:32,  6.20s/it]
  5%|▌         | 39/750 [04:44<1:13:19,  6.19s/it]
                                                  
{'loss': 0.1634, 'grad_norm': 0.0572248138487339, 'learning_rate': 0.00038397382389415225, 'epoch': 0.26}
+
  5%|▌         | 39/750 [04:44<1:13:19,  6.19s/it]
  5%|▌         | 40/750 [04:51<1:13:14,  6.19s/it]
                                                  
{'loss': 0.159, 'grad_norm': 0.05119703337550163, 'learning_rate': 0.00037914377220257757, 'epoch': 0.27}
+
  5%|▌         | 40/750 [04:51<1:13:14,  6.19s/it]
  5%|▌         | 41/750 [04:57<1:12:50,  6.16s/it]
                                                  
{'loss': 0.1657, 'grad_norm': 0.05322200059890747, 'learning_rate': 0.0003744915251898558, 'epoch': 0.27}
+
  5%|▌         | 41/750 [04:57<1:12:50,  6.16s/it]
  6%|▌         | 42/750 [05:03<1:12:43,  6.16s/it]
                                                  
{'loss': 0.1597, 'grad_norm': 0.054438743740320206, 'learning_rate': 0.00037000643495047747, 'epoch': 0.28}
+
  6%|▌         | 42/750 [05:03<1:12:43,  6.16s/it]
  6%|▌         | 43/750 [05:09<1:12:43,  6.17s/it]
                                                  
{'loss': 0.1728, 'grad_norm': 0.05853711813688278, 'learning_rate': 0.00036567872543061367, 'epoch': 0.29}
+
  6%|▌         | 43/750 [05:09<1:12:43,  6.17s/it]
  6%|▌         | 44/750 [05:15<1:12:24,  6.15s/it]
                                                  
{'loss': 0.165, 'grad_norm': 0.06103643774986267, 'learning_rate': 0.0003614994027406106, 'epoch': 0.29}
+
  6%|▌         | 44/750 [05:15<1:12:24,  6.15s/it]
  6%|▌         | 45/750 [05:21<1:12:19,  6.16s/it]
                                                  
{'loss': 0.1441, 'grad_norm': 0.051887478679418564, 'learning_rate': 0.0003574601764921203, 'epoch': 0.3}
+
  6%|▌         | 45/750 [05:21<1:12:19,  6.16s/it]
  6%|▌         | 46/750 [05:27<1:12:17,  6.16s/it]
                                                  
{'loss': 0.1581, 'grad_norm': 0.05508127063512802, 'learning_rate': 0.00035355339059327376, 'epoch': 0.31}
+
  6%|▌         | 46/750 [05:27<1:12:17,  6.16s/it]
  6%|▋         | 47/750 [05:34<1:12:14,  6.17s/it]
                                                  
{'loss': 0.1728, 'grad_norm': 0.05967516452074051, 'learning_rate': 0.0003497719621866722, 'epoch': 0.31}
+
  6%|▋         | 47/750 [05:34<1:12:14,  6.17s/it]
  6%|▋         | 48/750 [05:40<1:12:22,  6.19s/it]
                                                  
{'loss': 0.1695, 'grad_norm': 0.06013147160410881, 'learning_rate': 0.0003461093276215865, 'epoch': 0.32}
+
  6%|▋         | 48/750 [05:40<1:12:22,  6.19s/it]
  7%|▋         | 49/750 [05:46<1:12:12,  6.18s/it]
                                                  
{'loss': 0.167, 'grad_norm': 0.054595716297626495, 'learning_rate': 0.0003425593945223371, 'epoch': 0.33}
+
  7%|▋         | 49/750 [05:46<1:12:12,  6.18s/it]
  7%|▋         | 50/750 [05:52<1:12:08,  6.18s/it]
                                                  
{'loss': 0.1626, 'grad_norm': 0.053376030176877975, 'learning_rate': 0.0003391164991562634, 'epoch': 0.33}
+
  7%|▋         | 50/750 [05:52<1:12:08,  6.18s/it]
  7%|▋         | 51/750 [05:58<1:12:06,  6.19s/it]
                                                  
{'loss': 0.1445, 'grad_norm': 0.05673731490969658, 'learning_rate': 0.00033577536842242567, 'epoch': 0.34}
+
  7%|▋         | 51/750 [05:58<1:12:06,  6.19s/it]
  7%|▋         | 52/750 [06:05<1:11:41,  6.16s/it]
                                                  
{'loss': 0.1604, 'grad_norm': 0.05698046833276749, 'learning_rate': 0.0003325310858805882, 'epoch': 0.35}
+
  7%|▋         | 52/750 [06:05<1:11:41,  6.16s/it]
  7%|▋         | 53/750 [06:11<1:11:27,  6.15s/it]
                                                  
{'loss': 0.1709, 'grad_norm': 0.06058603525161743, 'learning_rate': 0.0003293790613225677, 'epoch': 0.35}
+
  7%|▋         | 53/750 [06:11<1:11:27,  6.15s/it]
  7%|▋         | 54/750 [06:17<1:12:18,  6.23s/it]
                                                  
{'loss': 0.1623, 'grad_norm': 0.05327736586332321, 'learning_rate': 0.0003263150034575203, 'epoch': 0.36}
+
  7%|▋         | 54/750 [06:17<1:12:18,  6.23s/it]
  7%|▋         | 55/750 [06:23<1:11:41,  6.19s/it]
                                                  
{'loss': 0.1548, 'grad_norm': 0.0600532628595829, 'learning_rate': 0.00032333489534143163, 'epoch': 0.37}
+
  7%|▋         | 55/750 [06:23<1:11:41,  6.19s/it]
  7%|▋         | 56/750 [06:29<1:11:30,  6.18s/it]
                                                  
{'loss': 0.1501, 'grad_norm': 0.060378916561603546, 'learning_rate': 0.0003204349722308279, 'epoch': 0.37}
+
  7%|▋         | 56/750 [06:29<1:11:30,  6.18s/it]
  8%|▊         | 57/750 [06:35<1:11:13,  6.17s/it]
                                                  
{'loss': 0.1564, 'grad_norm': 0.05655357241630554, 'learning_rate': 0.0003176117015830118, 'epoch': 0.38}
+
  8%|▊         | 57/750 [06:35<1:11:13,  6.17s/it]
  8%|▊         | 58/750 [06:42<1:11:05,  6.16s/it]
                                                  
{'loss': 0.1486, 'grad_norm': 0.053116921335458755, 'learning_rate': 0.00031486176496120126, 'epoch': 0.39}
+
  8%|▊         | 58/750 [06:42<1:11:05,  6.16s/it]
  8%|▊         | 59/750 [06:48<1:11:03,  6.17s/it]
                                                  
{'loss': 0.1519, 'grad_norm': 0.05973028391599655, 'learning_rate': 0.0003121820416337943, 'epoch': 0.39}
+
  8%|▊         | 59/750 [06:48<1:11:03,  6.17s/it]
  8%|▊         | 60/750 [06:54<1:11:16,  6.20s/it]
                                                  
{'loss': 0.1591, 'grad_norm': 0.06483177840709686, 'learning_rate': 0.00030956959368344517, 'epoch': 0.4}
+
  8%|▊         | 60/750 [06:54<1:11:16,  6.20s/it]
  8%|▊         | 61/750 [07:00<1:11:08,  6.20s/it]
                                                  
{'loss': 0.1669, 'grad_norm': 0.055687643587589264, 'learning_rate': 0.00030702165246439413, 'epoch': 0.41}
+
  8%|▊         | 61/750 [07:00<1:11:08,  6.20s/it]
  8%|▊         | 62/750 [07:06<1:10:57,  6.19s/it]
                                                  
{'loss': 0.1501, 'grad_norm': 0.05972687900066376, 'learning_rate': 0.0003045356062661163, 'epoch': 0.41}
+
  8%|▊         | 62/750 [07:06<1:10:57,  6.19s/it]
  8%|▊         | 63/750 [07:13<1:10:48,  6.18s/it]
                                                  
{'loss': 0.1674, 'grad_norm': 0.06020728126168251, 'learning_rate': 0.0003021089890583219, 'epoch': 0.42}
+
  8%|▊         | 63/750 [07:13<1:10:48,  6.18s/it]
  9%|▊         | 64/750 [07:19<1:10:29,  6.17s/it]
                                                  
{'loss': 0.1754, 'grad_norm': 0.06687033176422119, 'learning_rate': 0.00029973947020704494, 'epoch': 0.43}
+
  9%|▊         | 64/750 [07:19<1:10:29,  6.17s/it]
  9%|▊         | 65/750 [07:25<1:10:07,  6.14s/it]
                                                  
{'loss': 0.151, 'grad_norm': 0.05545530095696449, 'learning_rate': 0.0002974248450643262, 'epoch': 0.43}
+
  9%|▊         | 65/750 [07:25<1:10:07,  6.14s/it]
  9%|▉         | 66/750 [07:31<1:10:00,  6.14s/it]
                                                  
{'loss': 0.1599, 'grad_norm': 0.057525184005498886, 'learning_rate': 0.0002951630263451236, 'epoch': 0.44}
+
  9%|▉         | 66/750 [07:31<1:10:00,  6.14s/it]
  9%|▉         | 67/750 [07:37<1:09:58,  6.15s/it]
                                                  
{'loss': 0.1645, 'grad_norm': 0.057863749563694, 'learning_rate': 0.0002929520362147839, 'epoch': 0.45}
+
  9%|▉         | 67/750 [07:37<1:09:58,  6.15s/it]
  9%|▉         | 68/750 [07:43<1:09:48,  6.14s/it]
                                                  
{'loss': 0.1417, 'grad_norm': 0.05643998458981514, 'learning_rate': 0.00029078999901889984, 'epoch': 0.45}
+
  9%|▉         | 68/750 [07:43<1:09:48,  6.14s/it]
  9%|▉         | 69/750 [07:49<1:09:48,  6.15s/it]
                                                  
{'loss': 0.1557, 'grad_norm': 0.059055693447589874, 'learning_rate': 0.00028867513459481295, 'epoch': 0.46}
+
  9%|▉         | 69/750 [07:49<1:09:48,  6.15s/it]
  9%|▉         | 70/750 [07:56<1:09:35,  6.14s/it]
                                                  
{'loss': 0.1386, 'grad_norm': 0.05257362127304077, 'learning_rate': 0.0002866057521105554, 'epoch': 0.47}
+
  9%|▉         | 70/750 [07:56<1:09:35,  6.14s/it]
  9%|▉         | 71/750 [08:02<1:09:25,  6.14s/it]
                                                  
{'loss': 0.1755, 'grad_norm': 0.06722479313611984, 'learning_rate': 0.0002845802443827711, 'epoch': 0.47}
+
  9%|▉         | 71/750 [08:02<1:09:25,  6.14s/it]
 10%|▉         | 72/750 [08:08<1:09:39,  6.16s/it]
                                                  
{'loss': 0.1437, 'grad_norm': 0.05343697965145111, 'learning_rate': 0.0002825970826302195, 'epoch': 0.48}
+
 10%|▉         | 72/750 [08:08<1:09:39,  6.16s/it]
 10%|▉         | 73/750 [08:14<1:09:51,  6.19s/it]
                                                  
{'loss': 0.1471, 'grad_norm': 0.05955975130200386, 'learning_rate': 0.0002806548116239436, 'epoch': 0.49}
+
 10%|▉         | 73/750 [08:14<1:09:51,  6.19s/it]
 10%|▉         | 74/750 [08:20<1:09:28,  6.17s/it]
                                                  
{'loss': 0.1644, 'grad_norm': 0.06076746806502342, 'learning_rate': 0.00027875204519913876, 'epoch': 0.49}
+
 10%|▉         | 74/750 [08:20<1:09:28,  6.17s/it]
 10%|█         | 75/750 [08:26<1:09:06,  6.14s/it]
                                                  
{'loss': 0.1655, 'grad_norm': 0.06146124377846718, 'learning_rate': 0.00027688746209726916, 'epoch': 0.5}
+
 10%|█         | 75/750 [08:26<1:09:06,  6.14s/it]
 10%|█         | 76/750 [08:33<1:09:11,  6.16s/it]
                                                  
{'loss': 0.1672, 'grad_norm': 0.06172005087137222, 'learning_rate': 0.0002750598021100904, 'epoch': 0.51}
+
 10%|█         | 76/750 [08:33<1:09:11,  6.16s/it]
 10%|█         | 77/750 [08:39<1:09:09,  6.17s/it]
                                                  
{'loss': 0.1737, 'grad_norm': 0.059807997196912766, 'learning_rate': 0.0002732678625000106, 'epoch': 0.51}
+
 10%|█         | 77/750 [08:39<1:09:09,  6.17s/it]
 10%|█         | 78/750 [08:45<1:08:59,  6.16s/it]
                                                  
{'loss': 0.1678, 'grad_norm': 0.05595369264483452, 'learning_rate': 0.00027151049467368427, 'epoch': 0.52}
+
 10%|█         | 78/750 [08:45<1:08:59,  6.16s/it]
 11%|█         | 79/750 [08:51<1:08:44,  6.15s/it]
                                                  
{'loss': 0.1535, 'grad_norm': 0.05986161530017853, 'learning_rate': 0.00026978660108793815, 'epoch': 0.53}
+
 11%|█         | 79/750 [08:51<1:08:44,  6.15s/it]
 11%|█         | 80/750 [08:57<1:08:26,  6.13s/it]
                                                  
{'loss': 0.156, 'grad_norm': 0.05672367662191391, 'learning_rate': 0.00026809513236909023, 'epoch': 0.53}
+
 11%|█         | 80/750 [08:57<1:08:26,  6.13s/it]
 11%|█         | 81/750 [09:03<1:08:33,  6.15s/it]
                                                  
{'loss': 0.1497, 'grad_norm': 0.056865569204092026, 'learning_rate': 0.00026643508462848443, 'epoch': 0.54}
+
 11%|█         | 81/750 [09:03<1:08:33,  6.15s/it]
 11%|█         | 82/750 [09:09<1:08:31,  6.15s/it]
                                                  
{'loss': 0.1348, 'grad_norm': 0.05400577932596207, 'learning_rate': 0.00026480549695863983, 'epoch': 0.55}
+
 11%|█         | 82/750 [09:09<1:08:31,  6.15s/it]
 11%|█         | 83/750 [09:16<1:08:19,  6.15s/it]
                                                  
{'loss': 0.1785, 'grad_norm': 0.06244485825300217, 'learning_rate': 0.000263205449095825, 'epoch': 0.55}
+
 11%|█         | 83/750 [09:16<1:08:19,  6.15s/it]
 11%|█         | 84/750 [09:22<1:08:13,  6.15s/it]
                                                  
{'loss': 0.1529, 'grad_norm': 0.053567878901958466, 'learning_rate': 0.0002616340592361418, 'epoch': 0.56}
+
 11%|█         | 84/750 [09:22<1:08:13,  6.15s/it]
 11%|█▏        | 85/750 [09:28<1:07:54,  6.13s/it]
                                                  
{'loss': 0.1436, 'grad_norm': 0.05497294291853905, 'learning_rate': 0.00026009048199334285, 'epoch': 0.57}
+
 11%|█▏        | 85/750 [09:28<1:07:54,  6.13s/it]
 11%|█▏        | 86/750 [09:34<1:08:07,  6.16s/it]
                                                  
{'loss': 0.1542, 'grad_norm': 0.05579181760549545, 'learning_rate': 0.0002585739064876405, 'epoch': 0.57}
+
 11%|█▏        | 86/750 [09:34<1:08:07,  6.16s/it]
 12%|█▏        | 87/750 [09:40<1:08:10,  6.17s/it]
                                                  
{'loss': 0.1425, 'grad_norm': 0.05983961001038551, 'learning_rate': 0.0002570835545556902, 'epoch': 0.58}
+
 12%|█▏        | 87/750 [09:40<1:08:10,  6.17s/it]
 12%|█▏        | 88/750 [09:46<1:07:59,  6.16s/it]
                                                  
{'loss': 0.1533, 'grad_norm': 0.055703334510326385, 'learning_rate': 0.00025561867907277256, 'epoch': 0.59}
+
 12%|█▏        | 88/750 [09:46<1:07:59,  6.16s/it]
 12%|█▏        | 89/750 [09:52<1:07:40,  6.14s/it]
                                                  
{'loss': 0.1455, 'grad_norm': 0.05484815314412117, 'learning_rate': 0.0002541785623789578, 'epoch': 0.59}
+
 12%|█▏        | 89/750 [09:52<1:07:40,  6.14s/it]
 12%|█▏        | 90/750 [09:59<1:07:26,  6.13s/it]
                                                  
{'loss': 0.14, 'grad_norm': 0.054177530109882355, 'learning_rate': 0.00025276251480171836, 'epoch': 0.6}
+
 12%|█▏        | 90/750 [09:59<1:07:26,  6.13s/it]
 12%|█▏        | 91/750 [10:05<1:07:59,  6.19s/it]
                                                  
{'loss': 0.1706, 'grad_norm': 0.0572199672460556, 'learning_rate': 0.00025136987326808516, 'epoch': 0.61}
+
 12%|█▏        | 91/750 [10:05<1:07:59,  6.19s/it]
 12%|█▏        | 92/750 [10:11<1:08:18,  6.23s/it]
                                                  
{'loss': 0.155, 'grad_norm': 0.06946826726198196, 'learning_rate': 0.00025, 'epoch': 0.61}
+
 12%|█▏        | 92/750 [10:11<1:08:18,  6.23s/it]
 12%|█▏        | 93/750 [10:17<1:07:49,  6.19s/it]
                                                  
{'loss': 0.1685, 'grad_norm': 0.05900181084871292, 'learning_rate': 0.00024865228128703607, 'epoch': 0.62}
+
 12%|█▏        | 93/750 [10:17<1:07:49,  6.19s/it]
 13%|█▎        | 94/750 [10:23<1:07:39,  6.19s/it]
                                                  
{'loss': 0.1467, 'grad_norm': 0.05384255573153496, 'learning_rate': 0.0002473261263311206, 'epoch': 0.63}
+
 13%|█▎        | 94/750 [10:24<1:07:39,  6.19s/it]
 13%|█▎        | 95/750 [10:30<1:07:16,  6.16s/it]
                                                  
{'loss': 0.1498, 'grad_norm': 0.05746941640973091, 'learning_rate': 0.00024602096615832094, 'epoch': 0.63}
+
 13%|█▎        | 95/750 [10:30<1:07:16,  6.16s/it]
 13%|█▎        | 96/750 [10:36<1:07:09,  6.16s/it]
                                                  
{'loss': 0.1632, 'grad_norm': 0.05806390568614006, 'learning_rate': 0.0002447362525931402, 'epoch': 0.64}
+
 13%|█▎        | 96/750 [10:36<1:07:09,  6.16s/it]
 13%|█▎        | 97/750 [10:42<1:06:48,  6.14s/it]
                                                  
{'loss': 0.1416, 'grad_norm': 0.060687288641929626, 'learning_rate': 0.0002434714572911246, 'epoch': 0.65}
+
 13%|█▎        | 97/750 [10:42<1:06:48,  6.14s/it]
 13%|█▎        | 98/750 [10:48<1:06:44,  6.14s/it]
                                                  
{'loss': 0.1612, 'grad_norm': 0.06274425238370895, 'learning_rate': 0.00024222607082590244, 'epoch': 0.65}
+
 13%|█▎        | 98/750 [10:48<1:06:44,  6.14s/it]
 13%|█▎        | 99/750 [10:54<1:06:46,  6.15s/it]
                                                  
{'loss': 0.1535, 'grad_norm': 0.06144196167588234, 'learning_rate': 0.00024099960182707373, 'epoch': 0.66}
+
 13%|█▎        | 99/750 [10:54<1:06:46,  6.15s/it]
 13%|█▎        | 100/750 [11:00<1:06:44,  6.16s/it]
                                                   
{'loss': 0.1478, 'grad_norm': 0.056718356907367706, 'learning_rate': 0.00023979157616563598, 'epoch': 0.67}
+
 13%|█▎        | 100/750 [11:00<1:06:44,  6.16s/it]
 13%|█▎        | 101/750 [11:07<1:06:46,  6.17s/it]
                                                   
{'loss': 0.158, 'grad_norm': 0.05717349052429199, 'learning_rate': 0.00023860153618387902, 'epoch': 0.67}
+
 13%|█▎        | 101/750 [11:07<1:06:46,  6.17s/it]
 14%|█▎        | 102/750 [11:13<1:06:32,  6.16s/it]
                                                   
{'loss': 0.1442, 'grad_norm': 0.06308379769325256, 'learning_rate': 0.00023742903996690855, 'epoch': 0.68}
+
 14%|█▎        | 102/750 [11:13<1:06:32,  6.16s/it]
 14%|█▎        | 103/750 [11:19<1:06:22,  6.16s/it]
                                                   
{'loss': 0.1537, 'grad_norm': 0.057361382991075516, 'learning_rate': 0.00023627366065316423, 'epoch': 0.69}
+
 14%|█▎        | 103/750 [11:19<1:06:22,  6.16s/it]
 14%|█▍        | 104/750 [11:25<1:06:12,  6.15s/it]
                                                   
{'loss': 0.1548, 'grad_norm': 0.060229916125535965, 'learning_rate': 0.00023513498578149006, 'epoch': 0.69}
+
 14%|█▍        | 104/750 [11:25<1:06:12,  6.15s/it]
 14%|█▍        | 105/750 [11:31<1:06:08,  6.15s/it]
                                                   
{'loss': 0.1592, 'grad_norm': 0.058541201055049896, 'learning_rate': 0.0002340126166724879, 'epoch': 0.7}
+
 14%|█▍        | 105/750 [11:31<1:06:08,  6.15s/it]
 14%|█▍        | 106/750 [11:37<1:06:13,  6.17s/it]
                                                   
{'loss': 0.1622, 'grad_norm': 0.05861387029290199, 'learning_rate': 0.00023290616784204732, 'epoch': 0.71}
+
 14%|█▍        | 106/750 [11:37<1:06:13,  6.17s/it]
 14%|█▍        | 107/750 [11:43<1:06:01,  6.16s/it]
                                                   
{'loss': 0.1609, 'grad_norm': 0.06087273359298706, 'learning_rate': 0.00023181526644509276, 'epoch': 0.71}
+
 14%|█▍        | 107/750 [11:43<1:06:01,  6.16s/it]
 14%|█▍        | 108/750 [11:50<1:06:00,  6.17s/it]
                                                   
{'loss': 0.1532, 'grad_norm': 0.05830592289566994, 'learning_rate': 0.00023073955174772432, 'epoch': 0.72}
+
 14%|█▍        | 108/750 [11:50<1:06:00,  6.17s/it]
 15%|█▍        | 109/750 [11:56<1:05:48,  6.16s/it]
                                                   
{'loss': 0.1466, 'grad_norm': 0.05924879387021065, 'learning_rate': 0.0002296786746260559, 'epoch': 0.73}
+
 15%|█▍        | 109/750 [11:56<1:05:48,  6.16s/it]
 15%|█▍        | 110/750 [12:02<1:05:59,  6.19s/it]
                                                   
{'loss': 0.165, 'grad_norm': 0.06394900381565094, 'learning_rate': 0.00022863229709016894, 'epoch': 0.73}
+
 15%|█▍        | 110/750 [12:02<1:05:59,  6.19s/it]
 15%|█▍        | 111/750 [12:08<1:05:48,  6.18s/it]
                                                   
{'loss': 0.1457, 'grad_norm': 0.06064837798476219, 'learning_rate': 0.00022760009183170774, 'epoch': 0.74}
+
 15%|█▍        | 111/750 [12:08<1:05:48,  6.18s/it]
 15%|█▍        | 112/750 [12:14<1:05:25,  6.15s/it]
                                                   
{'loss': 0.1627, 'grad_norm': 0.0619843453168869, 'learning_rate': 0.00022658174179374143, 'epoch': 0.75}
+
 15%|█▍        | 112/750 [12:14<1:05:25,  6.15s/it]
 15%|█▌        | 113/750 [12:20<1:05:09,  6.14s/it]
                                                   
{'loss': 0.134, 'grad_norm': 0.05793182924389839, 'learning_rate': 0.0002255769397616086, 'epoch': 0.75}
+
 15%|█▌        | 113/750 [12:20<1:05:09,  6.14s/it]
 15%|█▌        | 114/750 [12:27<1:05:15,  6.16s/it]
                                                   
{'loss': 0.1453, 'grad_norm': 0.05908743664622307, 'learning_rate': 0.00022458538797354575, 'epoch': 0.76}
+
 15%|█▌        | 114/750 [12:27<1:05:15,  6.16s/it]
 15%|█▌        | 115/750 [12:33<1:04:53,  6.13s/it]
                                                   
{'loss': 0.1451, 'grad_norm': 0.05919326841831207, 'learning_rate': 0.00022360679774997898, 'epoch': 0.77}
+
 15%|█▌        | 115/750 [12:33<1:04:53,  6.13s/it]
 15%|█▌        | 116/750 [12:39<1:04:57,  6.15s/it]
                                                   
{'loss': 0.147, 'grad_norm': 0.059384964406490326, 'learning_rate': 0.00022264088914043033, 'epoch': 0.77}
+
 15%|█▌        | 116/750 [12:39<1:04:57,  6.15s/it]
 16%|█▌        | 117/750 [12:45<1:04:49,  6.14s/it]
                                                   
{'loss': 0.1534, 'grad_norm': 0.05859189108014107, 'learning_rate': 0.00022168739058705874, 'epoch': 0.78}
+
 16%|█▌        | 117/750 [12:45<1:04:49,  6.14s/it]
 16%|█▌        | 118/750 [12:51<1:04:46,  6.15s/it]
                                                   
{'loss': 0.1611, 'grad_norm': 0.05996308475732803, 'learning_rate': 0.00022074603860391704, 'epoch': 0.79}
+
 16%|█▌        | 118/750 [12:51<1:04:46,  6.15s/it]
 16%|█▌        | 119/750 [12:57<1:04:50,  6.17s/it]
                                                   
{'loss': 0.1639, 'grad_norm': 0.05672045424580574, 'learning_rate': 0.00021981657747106436, 'epoch': 0.79}
+
 16%|█▌        | 119/750 [12:57<1:04:50,  6.17s/it]
 16%|█▌        | 120/750 [13:04<1:04:43,  6.16s/it]
                                                   
{'loss': 0.1436, 'grad_norm': 0.05615617334842682, 'learning_rate': 0.00021889875894272825, 'epoch': 0.8}
+
 16%|█▌        | 120/750 [13:04<1:04:43,  6.16s/it]
 16%|█▌        | 121/750 [13:10<1:04:38,  6.17s/it]
                                                   
{'loss': 0.1641, 'grad_norm': 0.06566428393125534, 'learning_rate': 0.00021799234196875997, 'epoch': 0.81}
+
 16%|█▌        | 121/750 [13:10<1:04:38,  6.17s/it]
 16%|█▋        | 122/750 [13:16<1:04:38,  6.18s/it]
                                                   
{'loss': 0.151, 'grad_norm': 0.061154574155807495, 'learning_rate': 0.0002170970924286726, 'epoch': 0.81}
+
 16%|█▋        | 122/750 [13:16<1:04:38,  6.18s/it]
 16%|█▋        | 123/750 [13:22<1:04:22,  6.16s/it]
                                                   
{'loss': 0.1414, 'grad_norm': 0.05678664520382881, 'learning_rate': 0.00021621278287759676, 'epoch': 0.82}
+
 16%|█▋        | 123/750 [13:22<1:04:22,  6.16s/it]
 17%|█▋        | 124/750 [13:28<1:04:18,  6.16s/it]
                                                   
{'loss': 0.1417, 'grad_norm': 0.05800967290997505, 'learning_rate': 0.0002153391923035272, 'epoch': 0.83}
+
 17%|█▋        | 124/750 [13:28<1:04:18,  6.16s/it]
 17%|█▋        | 125/750 [13:34<1:03:58,  6.14s/it]
                                                   
{'loss': 0.1555, 'grad_norm': 0.06010492518544197, 'learning_rate': 0.00021447610589527216, 'epoch': 0.83}
+
 17%|█▋        | 125/750 [13:34<1:03:58,  6.14s/it]
 17%|█▋        | 126/750 [13:40<1:03:35,  6.12s/it]
                                                   
{'loss': 0.1514, 'grad_norm': 0.06327202916145325, 'learning_rate': 0.0002136233148205519, 'epoch': 0.84}
+
 17%|█▋        | 126/750 [13:40<1:03:35,  6.12s/it]
 17%|█▋        | 127/750 [13:46<1:03:27,  6.11s/it]
                                                   
{'loss': 0.1522, 'grad_norm': 0.05858540162444115, 'learning_rate': 0.00021278061601372693, 'epoch': 0.85}
+
 17%|█▋        | 127/750 [13:46<1:03:27,  6.11s/it]
 17%|█▋        | 128/750 [13:53<1:03:21,  6.11s/it]
                                                   
{'loss': 0.1742, 'grad_norm': 0.06486589461565018, 'learning_rate': 0.00021194781197266462, 'epoch': 0.85}
+
 17%|█▋        | 128/750 [13:53<1:03:21,  6.11s/it]
 17%|█▋        | 129/750 [13:59<1:03:55,  6.18s/it]
                                                   
{'loss': 0.1633, 'grad_norm': 0.06139097735285759, 'learning_rate': 0.00021112471056428407, 'epoch': 0.86}
+
 17%|█▋        | 129/750 [13:59<1:03:55,  6.18s/it]
 17%|█▋        | 130/750 [14:05<1:03:39,  6.16s/it]
                                                   
{'loss': 0.1576, 'grad_norm': 0.05948341637849808, 'learning_rate': 0.00021031112483834333, 'epoch': 0.87}
+
 17%|█▋        | 130/750 [14:05<1:03:39,  6.16s/it]
 17%|█▋        | 131/750 [14:11<1:03:31,  6.16s/it]
                                                   
{'loss': 0.1407, 'grad_norm': 0.05731433629989624, 'learning_rate': 0.00020950687284906043, 'epoch': 0.87}
+
 17%|█▋        | 131/750 [14:11<1:03:31,  6.16s/it]
 18%|█▊        | 132/750 [14:17<1:03:18,  6.15s/it]
                                                   
{'loss': 0.143, 'grad_norm': 0.06076508387923241, 'learning_rate': 0.00020871177748418049, 'epoch': 0.88}
+
 18%|█▊        | 132/750 [14:17<1:03:18,  6.15s/it]
 18%|█▊        | 133/750 [14:23<1:03:10,  6.14s/it]
                                                   
{'loss': 0.1372, 'grad_norm': 0.05646653473377228, 'learning_rate': 0.00020792566630112533, 'epoch': 0.89}
+
 18%|█▊        | 133/750 [14:23<1:03:10,  6.14s/it]
 18%|█▊        | 134/750 [14:30<1:03:17,  6.16s/it]
                                                   
{'loss': 0.1469, 'grad_norm': 0.05682455748319626, 'learning_rate': 0.00020714837136988075, 'epoch': 0.89}
+
 18%|█▊        | 134/750 [14:30<1:03:17,  6.16s/it]
 18%|█▊        | 135/750 [14:36<1:03:05,  6.16s/it]
                                                   
{'loss': 0.1644, 'grad_norm': 0.06654910743236542, 'learning_rate': 0.00020637972912229674, 'epoch': 0.9}
+
 18%|█▊        | 135/750 [14:36<1:03:05,  6.16s/it]
 18%|█▊        | 136/750 [14:42<1:02:48,  6.14s/it]
                                                   
{'loss': 0.1511, 'grad_norm': 0.060441434383392334, 'learning_rate': 0.00020561958020749358, 'epoch': 0.91}
+
 18%|█▊        | 136/750 [14:42<1:02:48,  6.14s/it]
 18%|█▊        | 137/750 [14:48<1:02:59,  6.16s/it]
                                                   
{'loss': 0.151, 'grad_norm': 0.059441886842250824, 'learning_rate': 0.00020486776935308305, 'epoch': 0.91}
+
 18%|█▊        | 137/750 [14:48<1:02:59,  6.16s/it]
 18%|█▊        | 138/750 [14:54<1:02:45,  6.15s/it]
                                                   
{'loss': 0.1521, 'grad_norm': 0.057412050664424896, 'learning_rate': 0.00020412414523193154, 'epoch': 0.92}
+
 18%|█▊        | 138/750 [14:54<1:02:45,  6.15s/it]
 19%|█▊        | 139/750 [15:00<1:02:39,  6.15s/it]
                                                   
{'loss': 0.1581, 'grad_norm': 0.05850718542933464, 'learning_rate': 0.00020338856033420402, 'epoch': 0.93}
+
 19%|█▊        | 139/750 [15:00<1:02:39,  6.15s/it]
 19%|█▊        | 140/750 [15:06<1:02:29,  6.15s/it]
                                                   
{'loss': 0.1527, 'grad_norm': 0.061566490679979324, 'learning_rate': 0.00020266087084444437, 'epoch': 0.93}
+
 19%|█▊        | 140/750 [15:07<1:02:29,  6.15s/it]
 19%|█▉        | 141/750 [15:13<1:02:17,  6.14s/it]
                                                   
{'loss': 0.145, 'grad_norm': 0.0557980015873909, 'learning_rate': 0.00020194093652345886, 'epoch': 0.94}
+
 19%|█▉        | 141/750 [15:13<1:02:17,  6.14s/it]
 19%|█▉        | 142/750 [15:19<1:02:07,  6.13s/it]
                                                   
{'loss': 0.1441, 'grad_norm': 0.057859282940626144, 'learning_rate': 0.00020122862059478237, 'epoch': 0.95}
+
 19%|█▉        | 142/750 [15:19<1:02:07,  6.13s/it]
 19%|█▉        | 143/750 [15:25<1:02:01,  6.13s/it]
                                                   
{'loss': 0.155, 'grad_norm': 0.0581962987780571, 'learning_rate': 0.00020052378963551984, 'epoch': 0.95}
+
 19%|█▉        | 143/750 [15:25<1:02:01,  6.13s/it]
 19%|█▉        | 144/750 [15:31<1:01:47,  6.12s/it]
                                                   
{'loss': 0.1472, 'grad_norm': 0.05818427726626396, 'learning_rate': 0.00019982631347136332, 'epoch': 0.96}
+
 19%|█▉        | 144/750 [15:31<1:01:47,  6.12s/it]
 19%|█▉        | 145/750 [15:37<1:01:48,  6.13s/it]
                                                   
{'loss': 0.142, 'grad_norm': 0.05695812404155731, 'learning_rate': 0.00019913606507559878, 'epoch': 0.97}
+
 19%|█▉        | 145/750 [15:37<1:01:48,  6.13s/it]
 19%|█▉        | 146/750 [15:43<1:01:45,  6.14s/it]
                                                   
{'loss': 0.1522, 'grad_norm': 0.056605711579322815, 'learning_rate': 0.00019845292047192357, 'epoch': 0.97}
+
 19%|█▉        | 146/750 [15:43<1:01:45,  6.14s/it]
 20%|█▉        | 147/750 [15:50<1:02:15,  6.19s/it]
                                                   
{'loss': 0.1492, 'grad_norm': 0.05666675046086311, 'learning_rate': 0.00019777675864090656, 'epoch': 0.98}
+
 20%|█▉        | 147/750 [15:50<1:02:15,  6.19s/it]
 20%|█▉        | 148/750 [15:56<1:02:20,  6.21s/it]
                                                   
{'loss': 0.1573, 'grad_norm': 0.057044558227062225, 'learning_rate': 0.00019710746142993, 'epoch': 0.99}
+
 20%|█▉        | 148/750 [15:56<1:02:20,  6.21s/it]
 20%|█▉        | 149/750 [16:02<1:01:59,  6.19s/it]
                                                   
{'loss': 0.1379, 'grad_norm': 0.05781971663236618, 'learning_rate': 0.00019644491346646172, 'epoch': 0.99}
+
 20%|█▉        | 149/750 [16:02<1:01:59,  6.19s/it]
 20%|██        | 150/750 [16:10<1:08:21,  6.84s/it]
                                                   
{'loss': 0.1702, 'grad_norm': 0.06060534715652466, 'learning_rate': 0.00019578900207451217, 'epoch': 1.0}
+
 20%|██        | 150/750 [16:10<1:08:21,  6.84s/it]
 20%|██        | 151/750 [16:25<1:32:17,  9.25s/it]
                                                   
{'loss': 0.1304, 'grad_norm': 0.052561718970537186, 'learning_rate': 0.00019513961719414, 'epoch': 1.01}
+
 20%|██        | 151/750 [16:25<1:32:17,  9.25s/it]
 20%|██        | 152/750 [16:31<1:22:54,  8.32s/it]
                                                   
{'loss': 0.1311, 'grad_norm': 0.0578678622841835, 'learning_rate': 0.00019449665130387477, 'epoch': 1.01}
+
 20%|██        | 152/750 [16:31<1:22:54,  8.32s/it]
 20%|██        | 153/750 [16:37<1:16:16,  7.67s/it]
                                                   
{'loss': 0.1374, 'grad_norm': 0.0587400384247303, 'learning_rate': 0.0001938599993459332, 'epoch': 1.02}
+
 20%|██        | 153/750 [16:37<1:16:16,  7.67s/it]
 21%|██        | 154/750 [16:44<1:11:25,  7.19s/it]
                                                   
{'loss': 0.1544, 'grad_norm': 0.06339883804321289, 'learning_rate': 0.00019322955865411052, 'epoch': 1.03}
+
 21%|██        | 154/750 [16:44<1:11:25,  7.19s/it]
 21%|██        | 155/750 [16:50<1:08:28,  6.90s/it]
                                                   
{'loss': 0.1225, 'grad_norm': 0.058893393725156784, 'learning_rate': 0.00019260522888423458, 'epoch': 1.03}
+
 21%|██        | 155/750 [16:50<1:08:28,  6.90s/it]
 21%|██        | 156/750 [16:56<1:06:00,  6.67s/it]
                                                   
{'loss': 0.1413, 'grad_norm': 0.06108240410685539, 'learning_rate': 0.00019198691194707612, 'epoch': 1.04}
+
 21%|██        | 156/750 [16:56<1:06:00,  6.67s/it]
 21%|██        | 157/750 [17:02<1:04:37,  6.54s/it]
                                                   
{'loss': 0.1347, 'grad_norm': 0.06023404374718666, 'learning_rate': 0.00019137451194361295, 'epoch': 1.05}
+
 21%|██        | 157/750 [17:02<1:04:37,  6.54s/it]
 21%|██        | 158/750 [17:08<1:03:18,  6.42s/it]
                                                   
{'loss': 0.1268, 'grad_norm': 0.059654634445905685, 'learning_rate': 0.00019076793510255105, 'epoch': 1.05}
+
 21%|██        | 158/750 [17:08<1:03:18,  6.42s/it]
 21%|██        | 159/750 [17:14<1:02:27,  6.34s/it]
                                                   
{'loss': 0.1579, 'grad_norm': 0.06982489675283432, 'learning_rate': 0.00019016708972001076, 'epoch': 1.06}
+
 21%|██        | 159/750 [17:14<1:02:27,  6.34s/it]
 21%|██▏       | 160/750 [17:21<1:01:57,  6.30s/it]
                                                   
{'loss': 0.145, 'grad_norm': 0.0596097931265831, 'learning_rate': 0.00018957188610128878, 'epoch': 1.07}
+
 21%|██▏       | 160/750 [17:21<1:01:57,  6.30s/it]
 21%|██▏       | 161/750 [17:27<1:01:22,  6.25s/it]
                                                   
{'loss': 0.1252, 'grad_norm': 0.06285068392753601, 'learning_rate': 0.0001889822365046136, 'epoch': 1.07}
+
 21%|██▏       | 161/750 [17:27<1:01:22,  6.25s/it]
 22%|██▏       | 162/750 [17:33<1:01:10,  6.24s/it]
                                                   
{'loss': 0.142, 'grad_norm': 0.06414437294006348, 'learning_rate': 0.00018839805508681298, 'epoch': 1.08}
+
 22%|██▏       | 162/750 [17:33<1:01:10,  6.24s/it]
 22%|██▏       | 163/750 [17:39<1:00:34,  6.19s/it]
                                                   
{'loss': 0.139, 'grad_norm': 0.06256502121686935, 'learning_rate': 0.00018781925785081761, 'epoch': 1.09}
+
 22%|██▏       | 163/750 [17:39<1:00:34,  6.19s/it]
 22%|██▏       | 164/750 [17:45<1:00:16,  6.17s/it]
                                                   
{'loss': 0.1392, 'grad_norm': 0.06551229953765869, 'learning_rate': 0.0001872457625949279, 'epoch': 1.09}
+
 22%|██▏       | 164/750 [17:45<1:00:16,  6.17s/it]
 22%|██▏       | 165/750 [17:51<59:56,  6.15s/it]  
                                                 
{'loss': 0.1397, 'grad_norm': 0.06680116802453995, 'learning_rate': 0.00018667748886377504, 'epoch': 1.1}
+
 22%|██▏       | 165/750 [17:51<59:56,  6.15s/it]
 22%|██▏       | 166/750 [17:58<1:00:05,  6.17s/it]
                                                   
{'loss': 0.125, 'grad_norm': 0.05875927954912186, 'learning_rate': 0.00018611435790090854, 'epoch': 1.11}
+
 22%|██▏       | 166/750 [17:58<1:00:05,  6.17s/it]
 22%|██▏       | 167/750 [18:04<1:00:41,  6.25s/it]
                                                   
{'loss': 0.1372, 'grad_norm': 0.06187672168016434, 'learning_rate': 0.00018555629260294813, 'epoch': 1.11}
+
 22%|██▏       | 167/750 [18:04<1:00:41,  6.25s/it]
 22%|██▏       | 168/750 [18:10<1:00:19,  6.22s/it]
                                                   
{'loss': 0.1294, 'grad_norm': 0.06508839130401611, 'learning_rate': 0.00018500321747523874, 'epoch': 1.12}
+
 22%|██▏       | 168/750 [18:10<1:00:19,  6.22s/it]
 23%|██▎       | 169/750 [18:16<1:00:02,  6.20s/it]
                                                   
{'loss': 0.1254, 'grad_norm': 0.06170894578099251, 'learning_rate': 0.00018445505858895076, 'epoch': 1.13}
+
 23%|██▎       | 169/750 [18:16<1:00:02,  6.20s/it]
 23%|██▎       | 170/750 [18:22<59:36,  6.17s/it]  
                                                 
{'loss': 0.1485, 'grad_norm': 0.06959165632724762, 'learning_rate': 0.0001839117435395704, 'epoch': 1.13}
+
 23%|██▎       | 170/750 [18:22<59:36,  6.17s/it]
 23%|██▎       | 171/750 [18:28<59:24,  6.16s/it]
                                                 
{'loss': 0.1299, 'grad_norm': 0.06467976421117783, 'learning_rate': 0.00018337320140672696, 'epoch': 1.14}
+
 23%|██▎       | 171/750 [18:28<59:24,  6.16s/it]
 23%|██▎       | 172/750 [18:35<59:44,  6.20s/it]
                                                 
{'loss': 0.151, 'grad_norm': 0.07470030337572098, 'learning_rate': 0.00018283936271530683, 'epoch': 1.15}
+
 23%|██▎       | 172/750 [18:35<59:44,  6.20s/it]
 23%|██▎       | 173/750 [18:41<59:29,  6.19s/it]
                                                 
{'loss': 0.1437, 'grad_norm': 0.06697074323892593, 'learning_rate': 0.0001823101593978053, 'epoch': 1.15}
+
 23%|██▎       | 173/750 [18:41<59:29,  6.19s/it]
 23%|██▎       | 174/750 [18:47<59:06,  6.16s/it]
                                                 
{'loss': 0.1317, 'grad_norm': 0.06739087402820587, 'learning_rate': 0.00018178552475787025, 'epoch': 1.16}
+
 23%|██▎       | 174/750 [18:47<59:06,  6.16s/it]
 23%|██▎       | 175/750 [18:53<59:00,  6.16s/it]
                                                 
{'loss': 0.1523, 'grad_norm': 0.07121624797582626, 'learning_rate': 0.00018126539343499314, 'epoch': 1.17}
+
 23%|██▎       | 175/750 [18:53<59:00,  6.16s/it]
 23%|██▎       | 176/750 [18:59<59:06,  6.18s/it]
                                                 
{'loss': 0.1193, 'grad_norm': 0.06330068409442902, 'learning_rate': 0.0001807497013703053, 'epoch': 1.17}
+
 23%|██▎       | 176/750 [18:59<59:06,  6.18s/it]
 24%|██▎       | 177/750 [19:06<58:43,  6.15s/it]
                                                 
{'loss': 0.1372, 'grad_norm': 0.06708869338035583, 'learning_rate': 0.00018023838577343806, 'epoch': 1.18}
+
 24%|██▎       | 177/750 [19:06<58:43,  6.15s/it]
 24%|██▎       | 178/750 [19:12<58:40,  6.16s/it]
                                                 
{'loss': 0.1289, 'grad_norm': 0.06606069207191467, 'learning_rate': 0.0001797313850904089, 'epoch': 1.19}
+
 24%|██▎       | 178/750 [19:12<58:40,  6.16s/it]
 24%|██▍       | 179/750 [19:18<58:28,  6.14s/it]
                                                 
{'loss': 0.136, 'grad_norm': 0.06582978367805481, 'learning_rate': 0.00017922863897249503, 'epoch': 1.19}
+
 24%|██▍       | 179/750 [19:18<58:28,  6.14s/it]
 24%|██▍       | 180/750 [19:24<58:11,  6.12s/it]
                                                 
{'loss': 0.1268, 'grad_norm': 0.06817764788866043, 'learning_rate': 0.00017873008824606016, 'epoch': 1.2}
+
 24%|██▍       | 180/750 [19:24<58:11,  6.12s/it]
 24%|██▍       | 181/750 [19:30<58:00,  6.12s/it]
                                                 
{'loss': 0.1308, 'grad_norm': 0.06883689761161804, 'learning_rate': 0.00017823567488329874, 'epoch': 1.21}
+
 24%|██▍       | 181/750 [19:30<58:00,  6.12s/it]
 24%|██▍       | 182/750 [19:36<57:52,  6.11s/it]
                                                 
{'loss': 0.1286, 'grad_norm': 0.0751621350646019, 'learning_rate': 0.00017774534197386607, 'epoch': 1.21}
+
 24%|██▍       | 182/750 [19:36<57:52,  6.11s/it]
 24%|██▍       | 183/750 [19:42<57:43,  6.11s/it]
                                                 
{'loss': 0.1321, 'grad_norm': 0.07165027409791946, 'learning_rate': 0.0001772590336973617, 'epoch': 1.22}
+
 24%|██▍       | 183/750 [19:42<57:43,  6.11s/it]
 25%|██▍       | 184/750 [19:48<57:34,  6.10s/it]
                                                 
{'loss': 0.1414, 'grad_norm': 0.0727408155798912, 'learning_rate': 0.00017677669529663688, 'epoch': 1.23}
+
 25%|██▍       | 184/750 [19:48<57:34,  6.10s/it]
 25%|██▍       | 185/750 [19:54<57:39,  6.12s/it]
                                                 
{'loss': 0.1329, 'grad_norm': 0.0777570828795433, 'learning_rate': 0.00017629827305189658, 'epoch': 1.23}
+
 25%|██▍       | 185/750 [19:54<57:39,  6.12s/it]
 25%|██▍       | 186/750 [20:01<58:02,  6.17s/it]
                                                 
{'loss': 0.1431, 'grad_norm': 0.07538975030183792, 'learning_rate': 0.0001758237142555681, 'epoch': 1.24}
+
 25%|██▍       | 186/750 [20:01<58:02,  6.17s/it]
 25%|██▍       | 187/750 [20:07<57:45,  6.16s/it]
                                                 
{'loss': 0.1563, 'grad_norm': 0.07506070286035538, 'learning_rate': 0.00017535296718791012, 'epoch': 1.25}
+
 25%|██▍       | 187/750 [20:07<57:45,  6.16s/it]
 25%|██▌       | 188/750 [20:13<57:41,  6.16s/it]
                                                 
{'loss': 0.1398, 'grad_norm': 0.07287327945232391, 'learning_rate': 0.0001748859810933361, 'epoch': 1.25}
+
 25%|██▌       | 188/750 [20:13<57:41,  6.16s/it]
 25%|██▌       | 189/750 [20:19<57:37,  6.16s/it]
                                                 
{'loss': 0.1322, 'grad_norm': 0.06793848425149918, 'learning_rate': 0.00017442270615742787, 'epoch': 1.26}
+
 25%|██▌       | 189/750 [20:19<57:37,  6.16s/it]
 25%|██▌       | 190/750 [20:25<57:17,  6.14s/it]
                                                 
{'loss': 0.1347, 'grad_norm': 0.07267748564481735, 'learning_rate': 0.00017396309348461487, 'epoch': 1.27}
+
 25%|██▌       | 190/750 [20:25<57:17,  6.14s/it]
 25%|██▌       | 191/750 [20:31<57:03,  6.13s/it]
                                                 
{'loss': 0.1402, 'grad_norm': 0.07143840938806534, 'learning_rate': 0.00017350709507649772, 'epoch': 1.27}
+
 25%|██▌       | 191/750 [20:31<57:03,  6.13s/it]
 26%|██▌       | 192/750 [20:37<56:53,  6.12s/it]
                                                 
{'loss': 0.1425, 'grad_norm': 0.07258402556180954, 'learning_rate': 0.00017305466381079324, 'epoch': 1.28}
+
 26%|██▌       | 192/750 [20:37<56:53,  6.12s/it]
 26%|██▌       | 193/750 [20:44<56:45,  6.11s/it]
                                                 
{'loss': 0.1391, 'grad_norm': 0.07381188124418259, 'learning_rate': 0.00017260575342088004, 'epoch': 1.29}
+
 26%|██▌       | 193/750 [20:44<56:45,  6.11s/it]
 26%|██▌       | 194/750 [20:50<56:50,  6.13s/it]
                                                 
{'loss': 0.1278, 'grad_norm': 0.06812915205955505, 'learning_rate': 0.0001721603184759251, 'epoch': 1.29}
+
 26%|██▌       | 194/750 [20:50<56:50,  6.13s/it]
 26%|██▌       | 195/750 [20:56<56:48,  6.14s/it]
                                                 
{'loss': 0.127, 'grad_norm': 0.07576236873865128, 'learning_rate': 0.0001717183143615715, 'epoch': 1.3}
+
 26%|██▌       | 195/750 [20:56<56:48,  6.14s/it]
 26%|██▌       | 196/750 [21:02<56:40,  6.14s/it]
                                                 
{'loss': 0.1098, 'grad_norm': 0.06439103931188583, 'learning_rate': 0.00017127969726116855, 'epoch': 1.31}
+
 26%|██▌       | 196/750 [21:02<56:40,  6.14s/it]
 26%|██▋       | 197/750 [21:08<56:40,  6.15s/it]
                                                 
{'loss': 0.1304, 'grad_norm': 0.07417545467615128, 'learning_rate': 0.0001708444241375271, 'epoch': 1.31}
+
 26%|██▋       | 197/750 [21:08<56:40,  6.15s/it]
 26%|██▋       | 198/750 [21:14<56:42,  6.16s/it]
                                                 
{'loss': 0.1438, 'grad_norm': 0.07219208776950836, 'learning_rate': 0.00017041245271518172, 'epoch': 1.32}
+
 26%|██▋       | 198/750 [21:14<56:42,  6.16s/it]
 27%|██▋       | 199/750 [21:21<56:34,  6.16s/it]
                                                 
{'loss': 0.1233, 'grad_norm': 0.06883670389652252, 'learning_rate': 0.00016998374146314418, 'epoch': 1.33}
+
 27%|██▋       | 199/750 [21:21<56:34,  6.16s/it]
 27%|██▋       | 200/750 [21:27<56:32,  6.17s/it]
                                                 
{'loss': 0.1383, 'grad_norm': 0.06966495513916016, 'learning_rate': 0.0001695582495781317, 'epoch': 1.33}
+
 27%|██▋       | 200/750 [21:27<56:32,  6.17s/it][INFO|trainer.py:3831] 2025-06-26 22:37:20,428 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-26 22:37:20,428 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-26 22:37:20,428 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:01<00:07,  1.09it/s]
+
 30%|███       | 3/10 [00:03<00:09,  1.31s/it]
+
 40%|████      | 4/10 [00:05<00:09,  1.52s/it]
+
 50%|█████     | 5/10 [00:07<00:08,  1.63s/it]
+
 60%|██████    | 6/10 [00:09<00:06,  1.70s/it]
+
 70%|███████   | 7/10 [00:11<00:05,  1.75s/it]
+
 80%|████████  | 8/10 [00:12<00:03,  1.78s/it]
+
 90%|█████████ | 9/10 [00:14<00:01,  1.80s/it]
+
100%|██████████| 10/10 [00:19<00:00,  2.71s/it]
                                                 
+
                                               
{'eval_loss': 0.14537057280540466, 'eval_accuracy': 0.2941094284318515, 'eval_runtime': 35.8238, 'eval_samples_per_second': 27.914, 'eval_steps_per_second': 0.279, 'epoch': 1.33}
+
 27%|██▋       | 200/750 [22:03<56:32,  6.17s/it]
+
100%|██████████| 10/10 [00:25<00:00,  2.71s/it]
+
                                               /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[INFO|trainer.py:3515] 2025-06-26 22:38:10,335 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-200
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d7e7a-160b9dd52291a7d445eee5d2;fe9dd48d-32d5-4e35-84ef-cc7070179a48)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-26 22:38:10,660 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-200/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-26 22:38:10,661 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-200/special_tokens_map.json
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[2025-06-26 22:38:11,963] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Checkpoint global_step200 is begin to save!
+[2025-06-26 22:38:11,987] [INFO] [logging.py:107:log_dist] [Rank 0] Saving model checkpoint: ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-200/global_step200/mp_rank_00_model_states.pt
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d7e7c-4cf14fd44a6694cb25981967;d7283c9f-fa90-444d-bbea-46bea298fcf5)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d7e7c-633ce74e1455bd3545548398;24759ea6-d0f3-4895-8d5b-e2055ad76032)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d7e7c-11dbdddf35976cb935949f09;2a54065e-894f-4983-abce-4f734f5b3ca7)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d7e7c-503f276f43c5a5f745d860f2;8523dab7-8c8a-4ab6-8eca-d1d7ee3da181)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+
 27%|██▋       | 201/750 [22:25<3:19:11, 21.77s/it]
                                                   
{'loss': 0.1394, 'grad_norm': 0.07412373274564743, 'learning_rate': 0.0001691359369682545, 'epoch': 1.34}
+
 27%|██▋       | 201/750 [22:25<3:19:11, 21.77s/it]
 27%|██▋       | 202/750 [22:31<2:35:41, 17.05s/it]
                                                   
{'loss': 0.1271, 'grad_norm': 0.07385868579149246, 'learning_rate': 0.00016871676423714827, 'epoch': 1.35}
+
 27%|██▋       | 202/750 [22:31<2:35:41, 17.05s/it]
 27%|██▋       | 203/750 [22:37<2:06:11, 13.84s/it]
                                                   
{'loss': 0.145, 'grad_norm': 0.0765007734298706, 'learning_rate': 0.00016830069266853705, 'epoch': 1.35}
+
 27%|██▋       | 203/750 [22:37<2:06:11, 13.84s/it]
 27%|██▋       | 204/750 [22:43<1:44:48, 11.52s/it]
                                                   
{'loss': 0.1422, 'grad_norm': 0.07773333042860031, 'learning_rate': 0.00016788768421121283, 'epoch': 1.36}
+
 27%|██▋       | 204/750 [22:43<1:44:48, 11.52s/it]
 27%|██▋       | 205/750 [22:50<1:30:14,  9.94s/it]
                                                   
{'loss': 0.1318, 'grad_norm': 0.07310941815376282, 'learning_rate': 0.00016747770146441848, 'epoch': 1.37}
+
 27%|██▋       | 205/750 [22:50<1:30:14,  9.94s/it]
 27%|██▋       | 206/750 [22:56<1:19:45,  8.80s/it]
                                                   
{'loss': 0.1334, 'grad_norm': 0.07561302930116653, 'learning_rate': 0.0001670707076636216, 'epoch': 1.37}
+
 27%|██▋       | 206/750 [22:56<1:19:45,  8.80s/it]
 28%|██▊       | 207/750 [23:02<1:12:12,  7.98s/it]
                                                   
{'loss': 0.1287, 'grad_norm': 0.06950889527797699, 'learning_rate': 0.00016666666666666666, 'epoch': 1.38}
+
 28%|██▊       | 207/750 [23:02<1:12:12,  7.98s/it]
 28%|██▊       | 208/750 [23:08<1:07:16,  7.45s/it]
                                                   
{'loss': 0.1292, 'grad_norm': 0.06586425751447678, 'learning_rate': 0.0001662655429402941, 'epoch': 1.39}
+
 28%|██▊       | 208/750 [23:08<1:07:16,  7.45s/it]
 28%|██▊       | 209/750 [23:14<1:03:46,  7.07s/it]
                                                   
{'loss': 0.1231, 'grad_norm': 0.06618046015501022, 'learning_rate': 0.00016586730154701388, 'epoch': 1.39}
+
 28%|██▊       | 209/750 [23:14<1:03:46,  7.07s/it]
 28%|██▊       | 210/750 [23:20<1:01:03,  6.78s/it]
                                                   
{'loss': 0.1312, 'grad_norm': 0.07627055048942566, 'learning_rate': 0.00016547190813232432, 'epoch': 1.4}
+
 28%|██▊       | 210/750 [23:20<1:01:03,  6.78s/it]
 28%|██▊       | 211/750 [23:27<59:23,  6.61s/it]  
                                                 
{'loss': 0.1287, 'grad_norm': 0.06810863316059113, 'learning_rate': 0.00016507932891226336, 'epoch': 1.41}
+
 28%|██▊       | 211/750 [23:27<59:23,  6.61s/it]
 28%|██▊       | 212/750 [23:33<58:15,  6.50s/it]
                                                 
{'loss': 0.1336, 'grad_norm': 0.06936858594417572, 'learning_rate': 0.00016468953066128386, 'epoch': 1.41}
+
 28%|██▊       | 212/750 [23:33<58:15,  6.50s/it]
 28%|██▊       | 213/750 [23:39<57:23,  6.41s/it]
                                                 
{'loss': 0.1513, 'grad_norm': 0.07809841632843018, 'learning_rate': 0.00016430248070044244, 'epoch': 1.42}
+
 28%|██▊       | 213/750 [23:39<57:23,  6.41s/it]
 29%|██▊       | 214/750 [23:45<56:29,  6.32s/it]
                                                 
{'loss': 0.1535, 'grad_norm': 0.07626284658908844, 'learning_rate': 0.0001639181468858914, 'epoch': 1.43}
+
 29%|██▊       | 214/750 [23:45<56:29,  6.32s/it]
 29%|██▊       | 215/750 [23:51<56:11,  6.30s/it]
                                                 
{'loss': 0.12, 'grad_norm': 0.06940948218107224, 'learning_rate': 0.00016353649759766664, 'epoch': 1.43}
+
 29%|██▊       | 215/750 [23:51<56:11,  6.30s/it]
 29%|██▉       | 216/750 [23:58<55:47,  6.27s/it]
                                                 
{'loss': 0.1202, 'grad_norm': 0.06957949697971344, 'learning_rate': 0.00016315750172876014, 'epoch': 1.44}
+
 29%|██▉       | 216/750 [23:58<55:47,  6.27s/it]
 29%|██▉       | 217/750 [24:04<55:27,  6.24s/it]
                                                 
{'loss': 0.13, 'grad_norm': 0.07661228626966476, 'learning_rate': 0.00016278112867447063, 'epoch': 1.45}
+
 29%|██▉       | 217/750 [24:04<55:27,  6.24s/it]
 29%|██▉       | 218/750 [24:10<55:00,  6.20s/it]
                                                 
{'loss': 0.123, 'grad_norm': 0.06977548450231552, 'learning_rate': 0.00016240734832202275, 'epoch': 1.45}
+
 29%|██▉       | 218/750 [24:10<55:00,  6.20s/it]
 29%|██▉       | 219/750 [24:16<54:44,  6.19s/it]
                                                 
{'loss': 0.118, 'grad_norm': 0.06987505406141281, 'learning_rate': 0.00016203613104044751, 'epoch': 1.46}
+
 29%|██▉       | 219/750 [24:16<54:44,  6.19s/it]
 29%|██▉       | 220/750 [24:22<54:36,  6.18s/it]
                                                 
{'loss': 0.1265, 'grad_norm': 0.07016448676586151, 'learning_rate': 0.00016166744767071581, 'epoch': 1.47}
+
 29%|██▉       | 220/750 [24:22<54:36,  6.18s/it]
 29%|██▉       | 221/750 [24:28<54:24,  6.17s/it]
                                                 
{'loss': 0.1315, 'grad_norm': 0.07573772966861725, 'learning_rate': 0.00016130126951611793, 'epoch': 1.47}
+
 29%|██▉       | 221/750 [24:28<54:24,  6.17s/it]
 30%|██▉       | 222/750 [24:35<54:48,  6.23s/it]
                                                 
{'loss': 0.1283, 'grad_norm': 0.07349874079227448, 'learning_rate': 0.0001609375683328815, 'epoch': 1.48}
+
 30%|██▉       | 222/750 [24:35<54:48,  6.23s/it]
 30%|██▉       | 223/750 [24:41<54:30,  6.21s/it]
                                                 
{'loss': 0.1388, 'grad_norm': 0.0752326026558876, 'learning_rate': 0.00016057631632102133, 'epoch': 1.49}
+
 30%|██▉       | 223/750 [24:41<54:30,  6.21s/it]
 30%|██▉       | 224/750 [24:47<54:10,  6.18s/it]
                                                 
{'loss': 0.1374, 'grad_norm': 0.07514145225286484, 'learning_rate': 0.00016021748611541394, 'epoch': 1.49}
+
 30%|██▉       | 224/750 [24:47<54:10,  6.18s/it]
 30%|███       | 225/750 [24:53<54:06,  6.18s/it]
                                                 
{'loss': 0.1247, 'grad_norm': 0.07078817486763, 'learning_rate': 0.00015986105077709064, 'epoch': 1.5}
+
 30%|███       | 225/750 [24:53<54:06,  6.18s/it]
 30%|███       | 226/750 [24:59<54:12,  6.21s/it]
                                                 
{'loss': 0.1386, 'grad_norm': 0.07227639853954315, 'learning_rate': 0.00015950698378474278, 'epoch': 1.51}
+
 30%|███       | 226/750 [24:59<54:12,  6.21s/it]
 30%|███       | 227/750 [25:06<54:02,  6.20s/it]
                                                 
{'loss': 0.1336, 'grad_norm': 0.07027120143175125, 'learning_rate': 0.00015915525902643283, 'epoch': 1.51}
+
 30%|███       | 227/750 [25:06<54:02,  6.20s/it]
 30%|███       | 228/750 [25:12<53:58,  6.20s/it]
                                                 
{'loss': 0.1321, 'grad_norm': 0.07193081825971603, 'learning_rate': 0.0001588058507915059, 'epoch': 1.52}
+
 30%|███       | 228/750 [25:12<53:58,  6.20s/it]
 31%|███       | 229/750 [25:18<53:31,  6.16s/it]
                                                 
{'loss': 0.1312, 'grad_norm': 0.07080845534801483, 'learning_rate': 0.00015845873376269562, 'epoch': 1.53}
+
 31%|███       | 229/750 [25:18<53:31,  6.16s/it]
 31%|███       | 230/750 [25:24<53:24,  6.16s/it]
                                                 
{'loss': 0.1313, 'grad_norm': 0.07075531780719757, 'learning_rate': 0.00015811388300841897, 'epoch': 1.53}
+
 31%|███       | 230/750 [25:24<53:24,  6.16s/it]
 31%|███       | 231/750 [25:30<53:22,  6.17s/it]
                                                 
{'loss': 0.1483, 'grad_norm': 0.07229692488908768, 'learning_rate': 0.00015777127397525472, 'epoch': 1.54}
+
 31%|███       | 231/750 [25:30<53:22,  6.17s/it]
 31%|███       | 232/750 [25:36<53:26,  6.19s/it]
                                                 
{'loss': 0.1437, 'grad_norm': 0.07453951984643936, 'learning_rate': 0.00015743088248060063, 'epoch': 1.55}
+
 31%|███       | 232/750 [25:36<53:26,  6.19s/it]
 31%|███       | 233/750 [25:43<53:13,  6.18s/it]
                                                 
{'loss': 0.1359, 'grad_norm': 0.07395069301128387, 'learning_rate': 0.0001570926847055038, 'epoch': 1.55}
+
 31%|███       | 233/750 [25:43<53:13,  6.18s/it]
 31%|███       | 234/750 [25:49<53:04,  6.17s/it]
                                                 
{'loss': 0.1353, 'grad_norm': 0.07679777592420578, 'learning_rate': 0.00015675665718766006, 'epoch': 1.56}
+
 31%|███       | 234/750 [25:49<53:04,  6.17s/it]
 31%|███▏      | 235/750 [25:55<52:51,  6.16s/it]
                                                 
{'loss': 0.1378, 'grad_norm': 0.07633081078529358, 'learning_rate': 0.00015642277681457702, 'epoch': 1.57}
+
 31%|███▏      | 235/750 [25:55<52:51,  6.16s/it]
 31%|███▏      | 236/750 [26:01<52:50,  6.17s/it]
                                                 
{'loss': 0.1424, 'grad_norm': 0.07358609139919281, 'learning_rate': 0.00015609102081689716, 'epoch': 1.57}
+
 31%|███▏      | 236/750 [26:01<52:50,  6.17s/it]
 32%|███▏      | 237/750 [26:07<52:25,  6.13s/it]
                                                 
{'loss': 0.1336, 'grad_norm': 0.07817543298006058, 'learning_rate': 0.00015576136676187527, 'epoch': 1.58}
+
 32%|███▏      | 237/750 [26:07<52:25,  6.13s/it]
 32%|███▏      | 238/750 [26:13<52:15,  6.12s/it]
                                                 
{'loss': 0.1366, 'grad_norm': 0.07987750321626663, 'learning_rate': 0.0001554337925470077, 'epoch': 1.59}
+
 32%|███▏      | 238/750 [26:13<52:15,  6.12s/it]
 32%|███▏      | 239/750 [26:19<52:18,  6.14s/it]
                                                 
{'loss': 0.1308, 'grad_norm': 0.07334929704666138, 'learning_rate': 0.00015510827639380736, 'epoch': 1.59}
+
 32%|███▏      | 239/750 [26:19<52:18,  6.14s/it]
 32%|███▏      | 240/750 [26:26<52:16,  6.15s/it]
                                                 
{'loss': 0.134, 'grad_norm': 0.07228033244609833, 'learning_rate': 0.00015478479684172258, 'epoch': 1.6}
+
 32%|███▏      | 240/750 [26:26<52:16,  6.15s/it]
 32%|███▏      | 241/750 [26:32<52:29,  6.19s/it]
                                                 
{'loss': 0.1277, 'grad_norm': 0.07475442439317703, 'learning_rate': 0.00015446333274219396, 'epoch': 1.61}
+
 32%|███▏      | 241/750 [26:32<52:29,  6.19s/it]
 32%|███▏      | 242/750 [26:38<52:20,  6.18s/it]
                                                 
{'loss': 0.1372, 'grad_norm': 0.07403514534235, 'learning_rate': 0.000154143863252847, 'epoch': 1.61}
+
 32%|███▏      | 242/750 [26:38<52:20,  6.18s/it]
 32%|███▏      | 243/750 [26:44<52:04,  6.16s/it]
                                                 
{'loss': 0.1285, 'grad_norm': 0.0721680298447609, 'learning_rate': 0.0001538263678318162, 'epoch': 1.62}
+
 32%|███▏      | 243/750 [26:44<52:04,  6.16s/it]
 33%|███▎      | 244/750 [26:50<52:00,  6.17s/it]
                                                 
{'loss': 0.1357, 'grad_norm': 0.0759200006723404, 'learning_rate': 0.00015351082623219707, 'epoch': 1.63}
+
 33%|███▎      | 244/750 [26:50<52:00,  6.17s/it]
 33%|███▎      | 245/750 [26:57<51:55,  6.17s/it]
                                                 
{'loss': 0.1371, 'grad_norm': 0.07299461215734482, 'learning_rate': 0.00015319721849662297, 'epoch': 1.63}
+
 33%|███▎      | 245/750 [26:57<51:55,  6.17s/it]
 33%|███▎      | 246/750 [27:03<52:00,  6.19s/it]
                                                 
{'loss': 0.1376, 'grad_norm': 0.07248148322105408, 'learning_rate': 0.0001528855249519633, 'epoch': 1.64}
+
 33%|███▎      | 246/750 [27:03<52:00,  6.19s/it]
 33%|███▎      | 247/750 [27:09<51:53,  6.19s/it]
                                                 
{'loss': 0.142, 'grad_norm': 0.07507827877998352, 'learning_rate': 0.00015257572620413985, 'epoch': 1.65}
+
 33%|███▎      | 247/750 [27:09<51:53,  6.19s/it]
 33%|███▎      | 248/750 [27:15<51:35,  6.17s/it]
                                                 
{'loss': 0.131, 'grad_norm': 0.07766193896532059, 'learning_rate': 0.00015226780313305814, 'epoch': 1.65}
+
 33%|███▎      | 248/750 [27:15<51:35,  6.17s/it]
 33%|███▎      | 249/750 [27:21<51:32,  6.17s/it]
                                                 
{'loss': 0.1243, 'grad_norm': 0.0721929520368576, 'learning_rate': 0.00015196173688765093, 'epoch': 1.66}
+
 33%|███▎      | 249/750 [27:21<51:32,  6.17s/it]
 33%|███▎      | 250/750 [27:27<51:16,  6.15s/it]
                                                 
{'loss': 0.1257, 'grad_norm': 0.0765080526471138, 'learning_rate': 0.00015165750888103104, 'epoch': 1.67}
+
 33%|███▎      | 250/750 [27:27<51:16,  6.15s/it]
 33%|███▎      | 251/750 [27:34<51:15,  6.16s/it]
                                                 
{'loss': 0.1342, 'grad_norm': 0.07611323893070221, 'learning_rate': 0.00015135510078575002, 'epoch': 1.67}
+
 33%|███▎      | 251/750 [27:34<51:15,  6.16s/it]
 34%|███▎      | 252/750 [27:40<51:08,  6.16s/it]
                                                 
{'loss': 0.1238, 'grad_norm': 0.07449932396411896, 'learning_rate': 0.00015105449452916094, 'epoch': 1.68}
+
 34%|███▎      | 252/750 [27:40<51:08,  6.16s/it]
 34%|███▎      | 253/750 [27:46<51:01,  6.16s/it]
                                                 
{'loss': 0.1289, 'grad_norm': 0.07692145556211472, 'learning_rate': 0.00015075567228888182, 'epoch': 1.69}
+
 34%|███▎      | 253/750 [27:46<51:01,  6.16s/it]
 34%|███▍      | 254/750 [27:52<50:47,  6.14s/it]
                                                 
{'loss': 0.1425, 'grad_norm': 0.07562893629074097, 'learning_rate': 0.0001504586164883572, 'epoch': 1.69}
+
 34%|███▍      | 254/750 [27:52<50:47,  6.14s/it]
 34%|███▍      | 255/750 [27:58<50:40,  6.14s/it]
                                                 
{'loss': 0.1381, 'grad_norm': 0.07521498948335648, 'learning_rate': 0.00015016330979251603, 'epoch': 1.7}
+
 34%|███▍      | 255/750 [27:58<50:40,  6.14s/it]
 34%|███▍      | 256/750 [28:04<50:26,  6.13s/it]
                                                 
{'loss': 0.1328, 'grad_norm': 0.07545516639947891, 'learning_rate': 0.00014986973510352247, 'epoch': 1.71}
+
 34%|███▍      | 256/750 [28:04<50:26,  6.13s/it]
 34%|███▍      | 257/750 [28:10<50:19,  6.12s/it]
                                                 
{'loss': 0.1375, 'grad_norm': 0.07514532655477524, 'learning_rate': 0.0001495778755566181, 'epoch': 1.71}
+
 34%|███▍      | 257/750 [28:10<50:19,  6.12s/it]
 34%|███▍      | 258/750 [28:17<50:28,  6.15s/it]
                                                 
{'loss': 0.1203, 'grad_norm': 0.07197596877813339, 'learning_rate': 0.0001492877145160524, 'epoch': 1.72}
+
 34%|███▍      | 258/750 [28:17<50:28,  6.15s/it]
 35%|███▍      | 259/750 [28:23<50:24,  6.16s/it]
                                                 
{'loss': 0.1345, 'grad_norm': 0.07074664533138275, 'learning_rate': 0.00014899923557110017, 'epoch': 1.73}
+
 35%|███▍      | 259/750 [28:23<50:24,  6.16s/it]
 35%|███▍      | 260/750 [28:29<50:45,  6.22s/it]
                                                 
{'loss': 0.1349, 'grad_norm': 0.07413626462221146, 'learning_rate': 0.0001487124225321631, 'epoch': 1.73}
+
 35%|███▍      | 260/750 [28:29<50:45,  6.22s/it]
 35%|███▍      | 261/750 [28:35<50:44,  6.23s/it]
                                                 
{'loss': 0.1207, 'grad_norm': 0.0763300210237503, 'learning_rate': 0.00014842725942695355, 'epoch': 1.74}
+
 35%|███▍      | 261/750 [28:35<50:44,  6.23s/it]
 35%|███▍      | 262/750 [28:41<50:25,  6.20s/it]
                                                 
{'loss': 0.1166, 'grad_norm': 0.07100091874599457, 'learning_rate': 0.0001481437304967584, 'epoch': 1.75}
+
 35%|███▍      | 262/750 [28:41<50:25,  6.20s/it]
 35%|███▌      | 263/750 [28:48<50:11,  6.18s/it]
                                                 
{'loss': 0.1319, 'grad_norm': 0.072751484811306, 'learning_rate': 0.00014786182019278145, 'epoch': 1.75}
+
 35%|███▌      | 263/750 [28:48<50:11,  6.18s/it]
 35%|███▌      | 264/750 [28:54<49:50,  6.15s/it]
                                                 
{'loss': 0.128, 'grad_norm': 0.07624773681163788, 'learning_rate': 0.0001475815131725618, 'epoch': 1.76}
+
 35%|███▌      | 264/750 [28:54<49:50,  6.15s/it]
 35%|███▌      | 265/750 [29:00<49:47,  6.16s/it]
                                                 
{'loss': 0.1368, 'grad_norm': 0.07538320869207382, 'learning_rate': 0.00014730279429646666, 'epoch': 1.77}
+
 35%|███▌      | 265/750 [29:00<49:47,  6.16s/it]
 35%|███▌      | 266/750 [29:06<49:37,  6.15s/it]
                                                 
{'loss': 0.1253, 'grad_norm': 0.07339820265769958, 'learning_rate': 0.0001470256486242569, 'epoch': 1.77}
+
 35%|███▌      | 266/750 [29:06<49:37,  6.15s/it]
 36%|███▌      | 267/750 [29:12<49:22,  6.13s/it]
                                                 
{'loss': 0.1305, 'grad_norm': 0.07588230818510056, 'learning_rate': 0.00014675006141172335, 'epoch': 1.78}
+
 36%|███▌      | 267/750 [29:12<49:22,  6.13s/it]
 36%|███▌      | 268/750 [29:18<49:12,  6.12s/it]
                                                 
{'loss': 0.1381, 'grad_norm': 0.07918047904968262, 'learning_rate': 0.00014647601810739195, 'epoch': 1.79}
+
 36%|███▌      | 268/750 [29:18<49:12,  6.12s/it]
 36%|███▌      | 269/750 [29:24<49:07,  6.13s/it]
                                                 
{'loss': 0.146, 'grad_norm': 0.07977352291345596, 'learning_rate': 0.0001462035043492969, 'epoch': 1.79}
+
 36%|███▌      | 269/750 [29:24<49:07,  6.13s/it]
 36%|███▌      | 270/750 [29:30<49:04,  6.13s/it]
                                                 
{'loss': 0.1335, 'grad_norm': 0.07707200944423676, 'learning_rate': 0.00014593250596181887, 'epoch': 1.8}
+
 36%|███▌      | 270/750 [29:30<49:04,  6.13s/it]
 36%|███▌      | 271/750 [29:37<49:01,  6.14s/it]
                                                 
{'loss': 0.1316, 'grad_norm': 0.07264542579650879, 'learning_rate': 0.00014566300895258813, 'epoch': 1.81}
+
 36%|███▌      | 271/750 [29:37<49:01,  6.14s/it]
 36%|███▋      | 272/750 [29:43<48:39,  6.11s/it]
                                                 
{'loss': 0.13, 'grad_norm': 0.08015874028205872, 'learning_rate': 0.00014539499950944992, 'epoch': 1.81}
+
 36%|███▋      | 272/750 [29:43<48:39,  6.11s/it]
 36%|███▋      | 273/750 [29:49<48:42,  6.13s/it]
                                                 
{'loss': 0.1249, 'grad_norm': 0.06936930865049362, 'learning_rate': 0.00014512846399749107, 'epoch': 1.82}
+
 36%|███▋      | 273/750 [29:49<48:42,  6.13s/it]
 37%|███▋      | 274/750 [29:55<48:50,  6.16s/it]
                                                 
{'loss': 0.1321, 'grad_norm': 0.07291866838932037, 'learning_rate': 0.00014486338895612657, 'epoch': 1.83}
+
 37%|███▋      | 274/750 [29:55<48:50,  6.16s/it]
 37%|███▋      | 275/750 [30:01<48:31,  6.13s/it]
                                                 
{'loss': 0.1096, 'grad_norm': 0.07074454426765442, 'learning_rate': 0.00014459976109624422, 'epoch': 1.83}
+
 37%|███▋      | 275/750 [30:01<48:31,  6.13s/it]
 37%|███▋      | 276/750 [30:07<48:25,  6.13s/it]
                                                 
{'loss': 0.1332, 'grad_norm': 0.07464369386434555, 'learning_rate': 0.00014433756729740648, 'epoch': 1.84}
+
 37%|███▋      | 276/750 [30:07<48:25,  6.13s/it]
 37%|███▋      | 277/750 [30:13<48:10,  6.11s/it]
                                                 
{'loss': 0.1485, 'grad_norm': 0.08192630112171173, 'learning_rate': 0.00014407679460510741, 'epoch': 1.85}
+
 37%|███▋      | 277/750 [30:13<48:10,  6.11s/it]
 37%|███▋      | 278/750 [30:20<48:28,  6.16s/it]
                                                 
{'loss': 0.1261, 'grad_norm': 0.06948667019605637, 'learning_rate': 0.00014381743022808492, 'epoch': 1.85}
+
 37%|███▋      | 278/750 [30:20<48:28,  6.16s/it]
 37%|███▋      | 279/750 [30:26<48:37,  6.19s/it]
                                                 
{'loss': 0.1213, 'grad_norm': 0.07278969883918762, 'learning_rate': 0.00014355946153568483, 'epoch': 1.86}
+
 37%|███▋      | 279/750 [30:26<48:37,  6.19s/it]
 37%|███▋      | 280/750 [30:32<48:32,  6.20s/it]
                                                 
{'loss': 0.1466, 'grad_norm': 0.08072081208229065, 'learning_rate': 0.0001433028760552777, 'epoch': 1.87}
+
 37%|███▋      | 280/750 [30:32<48:32,  6.20s/it]
 37%|███▋      | 281/750 [30:38<48:09,  6.16s/it]
                                                 
{'loss': 0.1494, 'grad_norm': 0.08002784103155136, 'learning_rate': 0.00014304766146972588, 'epoch': 1.87}
+
 37%|███▋      | 281/750 [30:38<48:09,  6.16s/it]
 38%|███▊      | 282/750 [30:44<48:02,  6.16s/it]
                                                 
{'loss': 0.133, 'grad_norm': 0.077474445104599, 'learning_rate': 0.0001427938056148999, 'epoch': 1.88}
+
 38%|███▊      | 282/750 [30:44<48:02,  6.16s/it]
 38%|███▊      | 283/750 [30:50<47:55,  6.16s/it]
                                                 
{'loss': 0.1345, 'grad_norm': 0.0726233720779419, 'learning_rate': 0.00014254129647724352, 'epoch': 1.89}
+
 38%|███▊      | 283/750 [30:50<47:55,  6.16s/it]
 38%|███▊      | 284/750 [30:57<47:47,  6.15s/it]
                                                 
{'loss': 0.1337, 'grad_norm': 0.07728508114814758, 'learning_rate': 0.00014229012219138556, 'epoch': 1.89}
+
 38%|███▊      | 284/750 [30:57<47:47,  6.15s/it]
 38%|███▊      | 285/750 [31:03<47:52,  6.18s/it]
                                                 
{'loss': 0.1377, 'grad_norm': 0.0696372240781784, 'learning_rate': 0.00014204027103779837, 'epoch': 1.9}
+
 38%|███▊      | 285/750 [31:03<47:52,  6.18s/it]
 38%|███▊      | 286/750 [31:09<47:47,  6.18s/it]
                                                 
{'loss': 0.1389, 'grad_norm': 0.07852566242218018, 'learning_rate': 0.0001417917314405008, 'epoch': 1.91}
+
 38%|███▊      | 286/750 [31:09<47:47,  6.18s/it]
 38%|███▊      | 287/750 [31:15<47:29,  6.15s/it]
                                                 
{'loss': 0.1234, 'grad_norm': 0.07629241049289703, 'learning_rate': 0.0001415444919648056, 'epoch': 1.91}
+
 38%|███▊      | 287/750 [31:15<47:29,  6.15s/it]
 38%|███▊      | 288/750 [31:21<47:25,  6.16s/it]
                                                 
{'loss': 0.1402, 'grad_norm': 0.08226976543664932, 'learning_rate': 0.00014129854131510975, 'epoch': 1.92}
+
 38%|███▊      | 288/750 [31:21<47:25,  6.16s/it]
 39%|███▊      | 289/750 [31:27<47:17,  6.16s/it]
                                                 
{'loss': 0.1187, 'grad_norm': 0.07371005415916443, 'learning_rate': 0.00014105386833272704, 'epoch': 1.93}
+
 39%|███▊      | 289/750 [31:27<47:17,  6.16s/it]
 39%|███▊      | 290/750 [31:34<47:07,  6.15s/it]
                                                 
{'loss': 0.1381, 'grad_norm': 0.08639305830001831, 'learning_rate': 0.0001408104619937615, 'epoch': 1.93}
+
 39%|███▊      | 290/750 [31:34<47:07,  6.15s/it]
 39%|███▉      | 291/750 [31:40<46:53,  6.13s/it]
                                                 
{'loss': 0.1224, 'grad_norm': 0.07645708322525024, 'learning_rate': 0.00014056831140702128, 'epoch': 1.94}
+
 39%|███▉      | 291/750 [31:40<46:53,  6.13s/it]
 39%|███▉      | 292/750 [31:46<46:47,  6.13s/it]
                                                 
{'loss': 0.1422, 'grad_norm': 0.07826756685972214, 'learning_rate': 0.0001403274058119718, 'epoch': 1.95}
+
 39%|███▉      | 292/750 [31:46<46:47,  6.13s/it]
 39%|███▉      | 293/750 [31:52<46:40,  6.13s/it]
                                                 
{'loss': 0.1377, 'grad_norm': 0.07914765924215317, 'learning_rate': 0.00014008773457672727, 'epoch': 1.95}
+
 39%|███▉      | 293/750 [31:52<46:40,  6.13s/it]
 39%|███▉      | 294/750 [31:58<46:29,  6.12s/it]
                                                 
{'loss': 0.1173, 'grad_norm': 0.07265656441450119, 'learning_rate': 0.00013984928719608012, 'epoch': 1.96}
+
 39%|███▉      | 294/750 [31:58<46:29,  6.12s/it]
 39%|███▉      | 295/750 [32:04<46:23,  6.12s/it]
                                                 
{'loss': 0.139, 'grad_norm': 0.07927590608596802, 'learning_rate': 0.0001396120532895667, 'epoch': 1.97}
+
 39%|███▉      | 295/750 [32:04<46:23,  6.12s/it]
 39%|███▉      | 296/750 [32:10<46:27,  6.14s/it]
                                                 
{'loss': 0.1361, 'grad_norm': 0.07623185217380524, 'learning_rate': 0.00013937602259956938, 'epoch': 1.97}
+
 39%|███▉      | 296/750 [32:10<46:27,  6.14s/it]
 40%|███▉      | 297/750 [32:17<46:54,  6.21s/it]
                                                 
{'loss': 0.1374, 'grad_norm': 0.07590196281671524, 'learning_rate': 0.00013914118498945362, 'epoch': 1.98}
+
 40%|███▉      | 297/750 [32:17<46:54,  6.21s/it]
 40%|███▉      | 298/750 [32:23<46:38,  6.19s/it]
                                                 
{'loss': 0.109, 'grad_norm': 0.07188297808170319, 'learning_rate': 0.00013890753044173962, 'epoch': 1.99}
+
 40%|███▉      | 298/750 [32:23<46:38,  6.19s/it]
 40%|███▉      | 299/750 [32:29<46:36,  6.20s/it]
                                                 
{'loss': 0.1296, 'grad_norm': 0.07715846598148346, 'learning_rate': 0.0001386750490563073, 'epoch': 1.99}
+
 40%|███▉      | 299/750 [32:29<46:36,  6.20s/it]
 40%|████      | 300/750 [32:37<51:18,  6.84s/it]
                                                 
{'loss': 0.1199, 'grad_norm': 0.07300840318202972, 'learning_rate': 0.00013844373104863458, 'epoch': 2.0}
+
 40%|████      | 300/750 [32:37<51:18,  6.84s/it]
 40%|████      | 301/750 [32:52<1:08:49,  9.20s/it]
                                                   
{'loss': 0.1165, 'grad_norm': 0.07021072506904602, 'learning_rate': 0.00013821356674806781, 'epoch': 2.01}
+
 40%|████      | 301/750 [32:52<1:08:49,  9.20s/it]
 40%|████      | 302/750 [32:58<1:01:38,  8.25s/it]
                                                   
{'loss': 0.1304, 'grad_norm': 0.07465282827615738, 'learning_rate': 0.00013798454659612338, 'epoch': 2.01}
+
 40%|████      | 302/750 [32:58<1:01:38,  8.25s/it]
 40%|████      | 303/750 [33:04<56:39,  7.61s/it]  
                                                 
{'loss': 0.1062, 'grad_norm': 0.07262144982814789, 'learning_rate': 0.00013775666114482078, 'epoch': 2.02}
+
 40%|████      | 303/750 [33:04<56:39,  7.61s/it]
 41%|████      | 304/750 [33:10<53:21,  7.18s/it]
                                                 
{'loss': 0.1178, 'grad_norm': 0.073135145008564, 'learning_rate': 0.0001375299010550452, 'epoch': 2.03}
+
 41%|████      | 304/750 [33:10<53:21,  7.18s/it]
 41%|████      | 305/750 [33:17<51:02,  6.88s/it]
                                                 
{'loss': 0.1167, 'grad_norm': 0.07653439044952393, 'learning_rate': 0.0001373042570949402, 'epoch': 2.03}
+
 41%|████      | 305/750 [33:17<51:02,  6.88s/it]
 41%|████      | 306/750 [33:23<49:12,  6.65s/it]
                                                 
{'loss': 0.1196, 'grad_norm': 0.08126599341630936, 'learning_rate': 0.00013707972013832906, 'epoch': 2.04}
+
 41%|████      | 306/750 [33:23<49:12,  6.65s/it]
 41%|████      | 307/750 [33:29<48:06,  6.52s/it]
                                                 
{'loss': 0.1064, 'grad_norm': 0.07932650297880173, 'learning_rate': 0.0001368562811631643, 'epoch': 2.05}
+
 41%|████      | 307/750 [33:29<48:06,  6.52s/it]
 41%|████      | 308/750 [33:35<47:21,  6.43s/it]
                                                 
{'loss': 0.1025, 'grad_norm': 0.07976703345775604, 'learning_rate': 0.0001366339312500053, 'epoch': 2.05}
+
 41%|████      | 308/750 [33:35<47:21,  6.43s/it]
 41%|████      | 309/750 [33:41<46:35,  6.34s/it]
                                                 
{'loss': 0.118, 'grad_norm': 0.09903603792190552, 'learning_rate': 0.00013641266158052266, 'epoch': 2.06}
+
 41%|████      | 309/750 [33:41<46:35,  6.34s/it]
 41%|████▏     | 310/750 [33:47<45:58,  6.27s/it]
                                                 
{'loss': 0.1226, 'grad_norm': 0.09472853690385818, 'learning_rate': 0.00013619246343602937, 'epoch': 2.07}
+
 41%|████▏     | 310/750 [33:47<45:58,  6.27s/it]
 41%|████▏     | 311/750 [33:54<45:36,  6.23s/it]
                                                 
{'loss': 0.1316, 'grad_norm': 0.09195204824209213, 'learning_rate': 0.00013597332819603774, 'epoch': 2.07}
+
 41%|████▏     | 311/750 [33:54<45:36,  6.23s/it]
 42%|████▏     | 312/750 [34:00<45:26,  6.22s/it]
                                                 
{'loss': 0.1228, 'grad_norm': 0.09129436314105988, 'learning_rate': 0.00013575524733684213, 'epoch': 2.08}
+
 42%|████▏     | 312/750 [34:00<45:26,  6.22s/it]
 42%|████▏     | 313/750 [34:06<45:08,  6.20s/it]
                                                 
{'loss': 0.0985, 'grad_norm': 0.08088590949773788, 'learning_rate': 0.00013553821243012613, 'epoch': 2.09}
+
 42%|████▏     | 313/750 [34:06<45:08,  6.20s/it]
 42%|████▏     | 314/750 [34:12<44:59,  6.19s/it]
                                                 
{'loss': 0.1147, 'grad_norm': 0.08690444380044937, 'learning_rate': 0.00013532221514159462, 'epoch': 2.09}
+
 42%|████▏     | 314/750 [34:12<44:59,  6.19s/it]
 42%|████▏     | 315/750 [34:18<44:47,  6.18s/it]
                                                 
{'loss': 0.1157, 'grad_norm': 0.08732669800519943, 'learning_rate': 0.00013510724722962957, 'epoch': 2.1}
+
 42%|████▏     | 315/750 [34:18<44:47,  6.18s/it]
 42%|████▏     | 316/750 [34:24<44:42,  6.18s/it]
                                                 
{'loss': 0.1173, 'grad_norm': 0.08484210819005966, 'learning_rate': 0.00013489330054396908, 'epoch': 2.11}
+
 42%|████▏     | 316/750 [34:24<44:42,  6.18s/it]
 42%|████▏     | 317/750 [34:31<45:09,  6.26s/it]
                                                 
{'loss': 0.1274, 'grad_norm': 0.08999501913785934, 'learning_rate': 0.0001346803670244095, 'epoch': 2.11}
+
 42%|████▏     | 317/750 [34:31<45:09,  6.26s/it]
 42%|████▏     | 318/750 [34:37<45:06,  6.27s/it]
                                                 
{'loss': 0.1123, 'grad_norm': 0.08566931635141373, 'learning_rate': 0.00013446843869953018, 'epoch': 2.12}
+
 42%|████▏     | 318/750 [34:37<45:06,  6.27s/it]
 43%|████▎     | 319/750 [34:43<44:39,  6.22s/it]
                                                 
{'loss': 0.1053, 'grad_norm': 0.08314401656389236, 'learning_rate': 0.0001342575076854399, 'epoch': 2.13}
+
 43%|████▎     | 319/750 [34:43<44:39,  6.22s/it]
 43%|████▎     | 320/750 [34:49<44:17,  6.18s/it]
                                                 
{'loss': 0.116, 'grad_norm': 0.08546454459428787, 'learning_rate': 0.00013404756618454512, 'epoch': 2.13}
+
 43%|████▎     | 320/750 [34:49<44:17,  6.18s/it]
 43%|████▎     | 321/750 [34:55<44:03,  6.16s/it]
                                                 
{'loss': 0.1171, 'grad_norm': 0.08344962447881699, 'learning_rate': 0.00013383860648433912, 'epoch': 2.14}
+
 43%|████▎     | 321/750 [34:55<44:03,  6.16s/it]
 43%|████▎     | 322/750 [35:02<43:57,  6.16s/it]
                                                 
{'loss': 0.1192, 'grad_norm': 0.09196280688047409, 'learning_rate': 0.0001336306209562122, 'epoch': 2.15}
+
 43%|████▎     | 322/750 [35:02<43:57,  6.16s/it]
 43%|████▎     | 323/750 [35:08<43:45,  6.15s/it]
                                                 
{'loss': 0.1134, 'grad_norm': 0.0936121791601181, 'learning_rate': 0.0001334236020542817, 'epoch': 2.15}
+
 43%|████▎     | 323/750 [35:08<43:45,  6.15s/it]
 43%|████▎     | 324/750 [35:14<43:35,  6.14s/it]
                                                 
{'loss': 0.1181, 'grad_norm': 0.0952591821551323, 'learning_rate': 0.00013321754231424221, 'epoch': 2.16}
+
 43%|████▎     | 324/750 [35:14<43:35,  6.14s/it]
 43%|████▎     | 325/750 [35:20<43:27,  6.13s/it]
                                                 
{'loss': 0.1233, 'grad_norm': 0.09443710744380951, 'learning_rate': 0.00013301243435223526, 'epoch': 2.17}
+
 43%|████▎     | 325/750 [35:20<43:27,  6.13s/it]
 43%|████▎     | 326/750 [35:26<43:25,  6.14s/it]
                                                 
{'loss': 0.1183, 'grad_norm': 0.09252423048019409, 'learning_rate': 0.00013280827086373782, 'epoch': 2.17}
+
 43%|████▎     | 326/750 [35:26<43:25,  6.14s/it]
 44%|████▎     | 327/750 [35:32<43:24,  6.16s/it]
                                                 
{'loss': 0.1144, 'grad_norm': 0.09512981027364731, 'learning_rate': 0.00013260504462246983, 'epoch': 2.18}
+
 44%|████▎     | 327/750 [35:32<43:24,  6.16s/it]
 44%|████▎     | 328/750 [35:38<43:25,  6.17s/it]
                                                 
{'loss': 0.0955, 'grad_norm': 0.08524834364652634, 'learning_rate': 0.00013240274847931992, 'epoch': 2.19}
+
 44%|████▎     | 328/750 [35:38<43:25,  6.17s/it]
 44%|████▍     | 329/750 [35:45<43:21,  6.18s/it]
                                                 
{'loss': 0.1118, 'grad_norm': 0.08862574398517609, 'learning_rate': 0.00013220137536128893, 'epoch': 2.19}
+
 44%|████▍     | 329/750 [35:45<43:21,  6.18s/it]
 44%|████▍     | 330/750 [35:51<43:11,  6.17s/it]
                                                 
{'loss': 0.1056, 'grad_norm': 0.08887834846973419, 'learning_rate': 0.00013200091827045155, 'epoch': 2.2}
+
 44%|████▍     | 330/750 [35:51<43:11,  6.17s/it]
 44%|████▍     | 331/750 [35:57<43:02,  6.16s/it]
                                                 
{'loss': 0.1124, 'grad_norm': 0.08845072239637375, 'learning_rate': 0.00013180137028293452, 'epoch': 2.21}
+
 44%|████▍     | 331/750 [35:57<43:02,  6.16s/it]
 44%|████▍     | 332/750 [36:03<42:46,  6.14s/it]
                                                 
{'loss': 0.109, 'grad_norm': 0.09361922740936279, 'learning_rate': 0.0001316027245479125, 'epoch': 2.21}
+
 44%|████▍     | 332/750 [36:03<42:46,  6.14s/it]
 44%|████▍     | 333/750 [36:09<42:38,  6.14s/it]
                                                 
{'loss': 0.1024, 'grad_norm': 0.0803227499127388, 'learning_rate': 0.00013140497428662003, 'epoch': 2.22}
+
 44%|████▍     | 333/750 [36:09<42:38,  6.14s/it]
 45%|████▍     | 334/750 [36:15<42:28,  6.13s/it]
                                                 
{'loss': 0.1415, 'grad_norm': 0.0981406420469284, 'learning_rate': 0.00013120811279137981, 'epoch': 2.23}
+
 45%|████▍     | 334/750 [36:15<42:28,  6.13s/it]
 45%|████▍     | 335/750 [36:22<42:48,  6.19s/it]
                                                 
{'loss': 0.104, 'grad_norm': 0.08900574594736099, 'learning_rate': 0.0001310121334246474, 'epoch': 2.23}
+
 45%|████▍     | 335/750 [36:22<42:48,  6.19s/it]
 45%|████▍     | 336/750 [36:28<42:37,  6.18s/it]
                                                 
{'loss': 0.0927, 'grad_norm': 0.08602827787399292, 'learning_rate': 0.0001308170296180709, 'epoch': 2.24}
+
 45%|████▍     | 336/750 [36:28<42:37,  6.18s/it]
 45%|████▍     | 337/750 [36:34<42:44,  6.21s/it]
                                                 
{'loss': 0.1185, 'grad_norm': 0.08875121921300888, 'learning_rate': 0.00013062279487156654, 'epoch': 2.25}
+
 45%|████▍     | 337/750 [36:34<42:44,  6.21s/it]
 45%|████▌     | 338/750 [36:40<42:25,  6.18s/it]
                                                 
{'loss': 0.119, 'grad_norm': 0.09354550391435623, 'learning_rate': 0.00013042942275240903, 'epoch': 2.25}
+
 45%|████▌     | 338/750 [36:40<42:25,  6.18s/it]
 45%|████▌     | 339/750 [36:46<42:14,  6.17s/it]
                                                 
{'loss': 0.1156, 'grad_norm': 0.09356021881103516, 'learning_rate': 0.0001302369068943367, 'epoch': 2.26}
+
 45%|████▌     | 339/750 [36:46<42:14,  6.17s/it]
 45%|████▌     | 340/750 [36:52<42:06,  6.16s/it]
                                                 
{'loss': 0.1035, 'grad_norm': 0.0859803631901741, 'learning_rate': 0.00013004524099667142, 'epoch': 2.27}
+
 45%|████▌     | 340/750 [36:52<42:06,  6.16s/it]
 45%|████▌     | 341/750 [36:59<42:04,  6.17s/it]
                                                 
{'loss': 0.1155, 'grad_norm': 0.0895763412117958, 'learning_rate': 0.00012985441882345223, 'epoch': 2.27}
+
 45%|████▌     | 341/750 [36:59<42:04,  6.17s/it]
 46%|████▌     | 342/750 [37:05<41:56,  6.17s/it]
                                                 
{'loss': 0.1122, 'grad_norm': 0.08926849812269211, 'learning_rate': 0.00012966443420258316, 'epoch': 2.28}
+
 46%|████▌     | 342/750 [37:05<41:56,  6.17s/it]
 46%|████▌     | 343/750 [37:11<41:49,  6.17s/it]
                                                 
{'loss': 0.1118, 'grad_norm': 0.08404593914747238, 'learning_rate': 0.00012947528102499512, 'epoch': 2.29}
+
 46%|████▌     | 343/750 [37:11<41:49,  6.17s/it]
 46%|████▌     | 344/750 [37:17<41:40,  6.16s/it]
                                                 
{'loss': 0.1137, 'grad_norm': 0.08592230081558228, 'learning_rate': 0.00012928695324382025, 'epoch': 2.29}
+
 46%|████▌     | 344/750 [37:17<41:40,  6.16s/it]
 46%|████▌     | 345/750 [37:23<41:33,  6.16s/it]
                                                 
{'loss': 0.1173, 'grad_norm': 0.09246755391359329, 'learning_rate': 0.00012909944487358055, 'epoch': 2.3}
+
 46%|████▌     | 345/750 [37:23<41:33,  6.16s/it]
 46%|████▌     | 346/750 [37:29<41:24,  6.15s/it]
                                                 
{'loss': 0.118, 'grad_norm': 0.09489478170871735, 'learning_rate': 0.0001289127499893885, 'epoch': 2.31}
+
 46%|████▌     | 346/750 [37:29<41:24,  6.15s/it]
 46%|████▋     | 347/750 [37:36<41:29,  6.18s/it]
                                                 
{'loss': 0.1126, 'grad_norm': 0.0884353369474411, 'learning_rate': 0.00012872686272616092, 'epoch': 2.31}
+
 46%|████▋     | 347/750 [37:36<41:29,  6.18s/it]
 46%|████▋     | 348/750 [37:42<41:21,  6.17s/it]
                                                 
{'loss': 0.1146, 'grad_norm': 0.09103699028491974, 'learning_rate': 0.0001285417772778451, 'epoch': 2.32}
+
 46%|████▋     | 348/750 [37:42<41:21,  6.17s/it]
 47%|████▋     | 349/750 [37:48<41:04,  6.14s/it]
                                                 
{'loss': 0.1137, 'grad_norm': 0.08928836137056351, 'learning_rate': 0.0001283574878966568, 'epoch': 2.33}
+
 47%|████▋     | 349/750 [37:48<41:04,  6.14s/it]
 47%|████▋     | 350/750 [37:54<41:05,  6.16s/it]
                                                 
{'loss': 0.1025, 'grad_norm': 0.08751604706048965, 'learning_rate': 0.00012817398889233115, 'epoch': 2.33}
+
 47%|████▋     | 350/750 [37:54<41:05,  6.16s/it]
 47%|████▋     | 351/750 [38:00<41:01,  6.17s/it]
                                                 
{'loss': 0.1135, 'grad_norm': 0.08962614089250565, 'learning_rate': 0.00012799127463138408, 'epoch': 2.34}
+
 47%|████▋     | 351/750 [38:00<41:01,  6.17s/it]
 47%|████▋     | 352/750 [38:06<40:56,  6.17s/it]
                                                 
{'loss': 0.1121, 'grad_norm': 0.09290952235460281, 'learning_rate': 0.00012780933953638628, 'epoch': 2.35}
+
 47%|████▋     | 352/750 [38:06<40:56,  6.17s/it]
 47%|████▋     | 353/750 [38:13<40:52,  6.18s/it]
                                                 
{'loss': 0.1069, 'grad_norm': 0.0916343405842781, 'learning_rate': 0.00012762817808524798, 'epoch': 2.35}
+
 47%|████▋     | 353/750 [38:13<40:52,  6.18s/it]
 47%|████▋     | 354/750 [38:19<40:57,  6.21s/it]
                                                 
{'loss': 0.1198, 'grad_norm': 0.09369686990976334, 'learning_rate': 0.00012744778481051504, 'epoch': 2.36}
+
 47%|████▋     | 354/750 [38:19<40:57,  6.21s/it]
 47%|████▋     | 355/750 [38:25<40:48,  6.20s/it]
                                                 
{'loss': 0.0985, 'grad_norm': 0.09005839377641678, 'learning_rate': 0.0001272681542986758, 'epoch': 2.37}
+
 47%|████▋     | 355/750 [38:25<40:48,  6.20s/it]
 47%|████▋     | 356/750 [38:31<40:37,  6.19s/it]
                                                 
{'loss': 0.1128, 'grad_norm': 0.09421628713607788, 'learning_rate': 0.0001270892811894789, 'epoch': 2.37}
+
 47%|████▋     | 356/750 [38:31<40:37,  6.19s/it]
 48%|████▊     | 357/750 [38:37<40:24,  6.17s/it]
                                                 
{'loss': 0.1111, 'grad_norm': 0.09863963723182678, 'learning_rate': 0.00012691116017526124, 'epoch': 2.38}
+
 48%|████▊     | 357/750 [38:37<40:24,  6.17s/it]
 48%|████▊     | 358/750 [38:44<40:15,  6.16s/it]
                                                 
{'loss': 0.1328, 'grad_norm': 0.1052733063697815, 'learning_rate': 0.00012673378600028677, 'epoch': 2.39}
+
 48%|████▊     | 358/750 [38:44<40:15,  6.16s/it]
 48%|████▊     | 359/750 [38:50<40:10,  6.16s/it]
                                                 
{'loss': 0.1253, 'grad_norm': 0.09568983316421509, 'learning_rate': 0.00012655715346009488, 'epoch': 2.39}
+
 48%|████▊     | 359/750 [38:50<40:10,  6.16s/it]
 48%|████▊     | 360/750 [38:56<40:02,  6.16s/it]
                                                 
{'loss': 0.122, 'grad_norm': 0.09231572598218918, 'learning_rate': 0.00012638125740085918, 'epoch': 2.4}
+
 48%|████▊     | 360/750 [38:56<40:02,  6.16s/it]
 48%|████▊     | 361/750 [39:02<39:48,  6.14s/it]
                                                 
{'loss': 0.1161, 'grad_norm': 0.0974225327372551, 'learning_rate': 0.0001262060927187558, 'epoch': 2.41}
+
 48%|████▊     | 361/750 [39:02<39:48,  6.14s/it]
 48%|████▊     | 362/750 [39:08<39:42,  6.14s/it]
                                                 
{'loss': 0.1106, 'grad_norm': 0.09488078206777573, 'learning_rate': 0.00012603165435934137, 'epoch': 2.41}
+
 48%|████▊     | 362/750 [39:08<39:42,  6.14s/it]
 48%|████▊     | 363/750 [39:14<39:39,  6.15s/it]
                                                 
{'loss': 0.1119, 'grad_norm': 0.0913190245628357, 'learning_rate': 0.00012585793731694053, 'epoch': 2.42}
+
 48%|████▊     | 363/750 [39:14<39:39,  6.15s/it]
 49%|████▊     | 364/750 [39:20<39:31,  6.14s/it]
                                                 
{'loss': 0.1076, 'grad_norm': 0.09292984008789062, 'learning_rate': 0.00012568493663404258, 'epoch': 2.43}
+
 49%|████▊     | 364/750 [39:20<39:31,  6.14s/it]
 49%|████▊     | 365/750 [39:27<39:29,  6.15s/it]
                                                 
{'loss': 0.1158, 'grad_norm': 0.09332066029310226, 'learning_rate': 0.00012551264740070717, 'epoch': 2.43}
+
 49%|████▊     | 365/750 [39:27<39:29,  6.15s/it]
 49%|████▉     | 366/750 [39:33<39:21,  6.15s/it]
                                                 
{'loss': 0.1338, 'grad_norm': 0.09921027719974518, 'learning_rate': 0.0001253410647539792, 'epoch': 2.44}
+
 49%|████▉     | 366/750 [39:33<39:21,  6.15s/it]
 49%|████▉     | 367/750 [39:39<39:04,  6.12s/it]
                                                 
{'loss': 0.1, 'grad_norm': 0.0893620178103447, 'learning_rate': 0.00012517018387731205, 'epoch': 2.45}
+
 49%|████▉     | 367/750 [39:39<39:04,  6.12s/it]
 49%|████▉     | 368/750 [39:45<38:58,  6.12s/it]
                                                 
{'loss': 0.1255, 'grad_norm': 0.10215151309967041, 'learning_rate': 0.000125, 'epoch': 2.45}
+
 49%|████▉     | 368/750 [39:45<38:58,  6.12s/it]
 49%|████▉     | 369/750 [39:51<39:01,  6.15s/it]
                                                 
{'loss': 0.1102, 'grad_norm': 0.0911877304315567, 'learning_rate': 0.0001248305083966186, 'epoch': 2.46}
+
 49%|████▉     | 369/750 [39:51<39:01,  6.15s/it]
 49%|████▉     | 370/750 [39:57<38:49,  6.13s/it]
                                                 
{'loss': 0.1097, 'grad_norm': 0.08814001083374023, 'learning_rate': 0.00012466170438647366, 'epoch': 2.47}
+
 49%|████▉     | 370/750 [39:57<38:49,  6.13s/it]
 49%|████▉     | 371/750 [40:03<38:40,  6.12s/it]
                                                 
{'loss': 0.1201, 'grad_norm': 0.09291444718837738, 'learning_rate': 0.00012449358333305828, 'epoch': 2.47}
+
 49%|████▉     | 371/750 [40:03<38:40,  6.12s/it]
 50%|████▉     | 372/750 [40:10<38:55,  6.18s/it]
                                                 
{'loss': 0.1249, 'grad_norm': 0.09924044460058212, 'learning_rate': 0.00012432614064351803, 'epoch': 2.48}
+
 50%|████▉     | 372/750 [40:10<38:55,  6.18s/it]
 50%|████▉     | 373/750 [40:16<38:41,  6.16s/it]
                                                 
{'loss': 0.1086, 'grad_norm': 0.09496409446001053, 'learning_rate': 0.00012415937176812403, 'epoch': 2.49}
+
 50%|████▉     | 373/750 [40:16<38:41,  6.16s/it]
 50%|████▉     | 374/750 [40:22<38:44,  6.18s/it]
                                                 
{'loss': 0.1074, 'grad_norm': 0.08924873918294907, 'learning_rate': 0.0001239932721997534, 'epoch': 2.49}
+
 50%|████▉     | 374/750 [40:22<38:44,  6.18s/it]
 50%|█████     | 375/750 [40:28<38:29,  6.16s/it]
                                                 
{'loss': 0.0993, 'grad_norm': 0.08711589127779007, 'learning_rate': 0.00012382783747337805, 'epoch': 2.5}
+
 50%|█████     | 375/750 [40:28<38:29,  6.16s/it]
 50%|█████     | 376/750 [40:34<38:21,  6.15s/it]
                                                 
{'loss': 0.1344, 'grad_norm': 0.10486136376857758, 'learning_rate': 0.0001236630631655603, 'epoch': 2.51}
+
 50%|█████     | 376/750 [40:34<38:21,  6.15s/it]
 50%|█████     | 377/750 [40:40<38:09,  6.14s/it]
                                                 
{'loss': 0.1127, 'grad_norm': 0.09214172512292862, 'learning_rate': 0.0001234989448939562, 'epoch': 2.51}
+
 50%|█████     | 377/750 [40:40<38:09,  6.14s/it]
 50%|█████     | 378/750 [40:46<38:01,  6.13s/it]
                                                 
{'loss': 0.1176, 'grad_norm': 0.0968175157904625, 'learning_rate': 0.00012333547831682581, 'epoch': 2.52}
+
 50%|█████     | 378/750 [40:46<38:01,  6.13s/it]
 51%|█████     | 379/750 [40:53<37:49,  6.12s/it]
                                                 
{'loss': 0.1094, 'grad_norm': 0.09431985020637512, 'learning_rate': 0.00012317265913255117, 'epoch': 2.53}
+
 51%|█████     | 379/750 [40:53<37:49,  6.12s/it]
 51%|█████     | 380/750 [40:59<37:43,  6.12s/it]
                                                 
{'loss': 0.1129, 'grad_norm': 0.09073186665773392, 'learning_rate': 0.00012301048307916047, 'epoch': 2.53}
+
 51%|█████     | 380/750 [40:59<37:43,  6.12s/it]
 51%|█████     | 381/750 [41:05<37:40,  6.12s/it]
                                                 
{'loss': 0.1104, 'grad_norm': 0.09180307388305664, 'learning_rate': 0.00012284894593385964, 'epoch': 2.54}
+
 51%|█████     | 381/750 [41:05<37:40,  6.12s/it]
 51%|█████     | 382/750 [41:11<37:37,  6.13s/it]
                                                 
{'loss': 0.1097, 'grad_norm': 0.09660235047340393, 'learning_rate': 0.00012268804351257058, 'epoch': 2.55}
+
 51%|█████     | 382/750 [41:11<37:37,  6.13s/it]
 51%|█████     | 383/750 [41:17<37:30,  6.13s/it]
                                                 
{'loss': 0.1135, 'grad_norm': 0.0907338410615921, 'learning_rate': 0.00012252777166947586, 'epoch': 2.55}
+
 51%|█████     | 383/750 [41:17<37:30,  6.13s/it]
 51%|█████     | 384/750 [41:23<37:20,  6.12s/it]
                                                 
{'loss': 0.1117, 'grad_norm': 0.09390397369861603, 'learning_rate': 0.0001223681262965701, 'epoch': 2.56}
+
 51%|█████     | 384/750 [41:23<37:20,  6.12s/it]
 51%|█████▏    | 385/750 [41:29<37:20,  6.14s/it]
                                                 
{'loss': 0.1106, 'grad_norm': 0.08993766456842422, 'learning_rate': 0.00012220910332321784, 'epoch': 2.57}
+
 51%|█████▏    | 385/750 [41:29<37:20,  6.14s/it]
 51%|█████▏    | 386/750 [41:35<37:13,  6.14s/it]
                                                 
{'loss': 0.1253, 'grad_norm': 0.09967799484729767, 'learning_rate': 0.00012205069871571739, 'epoch': 2.57}
+
 51%|█████▏    | 386/750 [41:35<37:13,  6.14s/it]
 52%|█████▏    | 387/750 [41:42<37:07,  6.14s/it]
                                                 
{'loss': 0.1179, 'grad_norm': 0.09442531317472458, 'learning_rate': 0.00012189290847687124, 'epoch': 2.58}
+
 52%|█████▏    | 387/750 [41:42<37:07,  6.14s/it]
 52%|█████▏    | 388/750 [41:48<37:01,  6.14s/it]
                                                 
{'loss': 0.1059, 'grad_norm': 0.0897684171795845, 'learning_rate': 0.0001217357286455623, 'epoch': 2.59}
+
 52%|█████▏    | 388/750 [41:48<37:01,  6.14s/it]
 52%|█████▏    | 389/750 [41:54<36:58,  6.15s/it]
                                                 
{'loss': 0.1054, 'grad_norm': 0.0885215774178505, 'learning_rate': 0.00012157915529633644, 'epoch': 2.59}
+
 52%|█████▏    | 389/750 [41:54<36:58,  6.15s/it]
 52%|█████▏    | 390/750 [42:00<37:02,  6.17s/it]
                                                 
{'loss': 0.1091, 'grad_norm': 0.09066983312368393, 'learning_rate': 0.00012142318453899053, 'epoch': 2.6}
+
 52%|█████▏    | 390/750 [42:00<37:02,  6.17s/it]
 52%|█████▏    | 391/750 [42:06<36:59,  6.18s/it]
                                                 
{'loss': 0.1095, 'grad_norm': 0.0868227556347847, 'learning_rate': 0.00012126781251816648, 'epoch': 2.61}
+
 52%|█████▏    | 391/750 [42:06<36:59,  6.18s/it]
 52%|█████▏    | 392/750 [42:13<37:11,  6.23s/it]
                                                 
{'loss': 0.1138, 'grad_norm': 0.10002143681049347, 'learning_rate': 0.00012111303541295122, 'epoch': 2.61}
+
 52%|█████▏    | 392/750 [42:13<37:11,  6.23s/it]
 52%|█████▏    | 393/750 [42:19<37:00,  6.22s/it]
                                                 
{'loss': 0.1204, 'grad_norm': 0.09589198231697083, 'learning_rate': 0.00012095884943648174, 'epoch': 2.62}
+
 52%|█████▏    | 393/750 [42:19<37:00,  6.22s/it]
 53%|█████▎    | 394/750 [42:25<36:37,  6.17s/it]
                                                 
{'loss': 0.1063, 'grad_norm': 0.09234515577554703, 'learning_rate': 0.0001208052508355561, 'epoch': 2.63}
+
 53%|█████▎    | 394/750 [42:25<36:37,  6.17s/it]
 53%|█████▎    | 395/750 [42:31<36:34,  6.18s/it]
                                                 
{'loss': 0.1007, 'grad_norm': 0.08737505972385406, 'learning_rate': 0.0001206522358902497, 'epoch': 2.63}
+
 53%|█████▎    | 395/750 [42:31<36:34,  6.18s/it]
 53%|█████▎    | 396/750 [42:37<36:18,  6.15s/it]
                                                 
{'loss': 0.0948, 'grad_norm': 0.09216974675655365, 'learning_rate': 0.00012049980091353687, 'epoch': 2.64}
+
 53%|█████▎    | 396/750 [42:37<36:18,  6.15s/it]
 53%|█████▎    | 397/750 [42:43<36:01,  6.12s/it]
                                                 
{'loss': 0.1093, 'grad_norm': 0.09234543144702911, 'learning_rate': 0.00012034794225091773, 'epoch': 2.65}
+
 53%|█████▎    | 397/750 [42:43<36:01,  6.12s/it]
 53%|█████▎    | 398/750 [42:49<35:58,  6.13s/it]
                                                 
{'loss': 0.1036, 'grad_norm': 0.08711022138595581, 'learning_rate': 0.00012019665628005017, 'epoch': 2.65}
+
 53%|█████▎    | 398/750 [42:49<35:58,  6.13s/it]
 53%|█████▎    | 399/750 [42:56<36:06,  6.17s/it]
                                                 
{'loss': 0.1233, 'grad_norm': 0.09455020725727081, 'learning_rate': 0.00012004593941038698, 'epoch': 2.66}
+
 53%|█████▎    | 399/750 [42:56<36:06,  6.17s/it]
 53%|█████▎    | 400/750 [43:02<35:55,  6.16s/it]
                                                 
{'loss': 0.1049, 'grad_norm': 0.09240896999835968, 'learning_rate': 0.00011989578808281799, 'epoch': 2.67}
+
 53%|█████▎    | 400/750 [43:02<35:55,  6.16s/it][INFO|trainer.py:3831] 2025-06-26 22:58:55,503 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-26 22:58:55,503 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-26 22:58:55,503 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:01<00:07,  1.08it/s]
+
 30%|███       | 3/10 [00:03<00:09,  1.31s/it]
+
 40%|████      | 4/10 [00:05<00:09,  1.51s/it]
+
 50%|█████     | 5/10 [00:07<00:08,  1.64s/it]
+
 60%|██████    | 6/10 [00:09<00:06,  1.70s/it]
+
 70%|███████   | 7/10 [00:11<00:05,  1.75s/it]
+
 80%|████████  | 8/10 [00:12<00:03,  1.78s/it]
+
 90%|█████████ | 9/10 [00:14<00:01,  1.80s/it]
+
100%|██████████| 10/10 [00:18<00:00,  2.40s/it]
                                                 
+
                                               
{'eval_loss': 0.1427709013223648, 'eval_accuracy': 0.2945705911089399, 'eval_runtime': 34.3156, 'eval_samples_per_second': 29.141, 'eval_steps_per_second': 0.291, 'epoch': 2.67}
+
 53%|█████▎    | 400/750 [43:36<35:55,  6.16s/it]
+
100%|██████████| 10/10 [00:24<00:00,  2.40s/it]
+
                                               /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[INFO|trainer.py:3515] 2025-06-26 22:59:43,612 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-400
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8387-3dc399f979fd808b3e40d1a2;51a5d680-d275-4423-918f-92815c6fe90c)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-26 22:59:43,933 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-400/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-26 22:59:43,933 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-400/special_tokens_map.json
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[2025-06-26 22:59:45,063] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Checkpoint global_step400 is begin to save!
+[2025-06-26 22:59:45,087] [INFO] [logging.py:107:log_dist] [Rank 0] Saving model checkpoint: ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8389-354de8d856eb7deb69298282;c5eb42b8-c68b-48b6-84a0-0efd50b13ded)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8389-6b420b49284924e9109d98ab;710ea7fd-43a0-4441-a437-fba37f707ab5)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8389-240d6f6d4a71fc21587f1fd1;1f770f79-38c1-4e71-9f9b-1f6e7b7ae9eb)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8389-25eda2446fa0bd79014f3091;bcb4bb3c-03da-4620-9f49-5bdd0f6864ae)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+
 53%|█████▎    | 401/750 [43:58<2:03:08, 21.17s/it]
                                                   
{'loss': 0.1254, 'grad_norm': 0.09577213227748871, 'learning_rate': 0.00011974619876931687, 'epoch': 2.67}
+
 53%|█████▎    | 401/750 [43:58<2:03:08, 21.17s/it]
 54%|█████▎    | 402/750 [44:04<1:36:30, 16.64s/it]
                                                   
{'loss': 0.1137, 'grad_norm': 0.10269856452941895, 'learning_rate': 0.0001195971679725932, 'epoch': 2.68}
+
 54%|█████▎    | 402/750 [44:04<1:36:30, 16.64s/it]
 54%|█████▎    | 403/750 [44:10<1:17:53, 13.47s/it]
                                                   
{'loss': 0.1142, 'grad_norm': 0.09806156903505325, 'learning_rate': 0.00011944869222574892, 'epoch': 2.69}
+
 54%|█████▎    | 403/750 [44:10<1:17:53, 13.47s/it]
 54%|█████▍    | 404/750 [44:16<1:05:08, 11.30s/it]
                                                   
{'loss': 0.1124, 'grad_norm': 0.0915805771946907, 'learning_rate': 0.00011930076809193951, 'epoch': 2.69}
+
 54%|█████▍    | 404/750 [44:16<1:05:08, 11.30s/it]
 54%|█████▍    | 405/750 [44:22<55:55,  9.73s/it]  
                                                 
{'loss': 0.1137, 'grad_norm': 0.10149909555912018, 'learning_rate': 0.0001191533921640401, 'epoch': 2.7}
+
 54%|█████▍    | 405/750 [44:22<55:55,  9.73s/it]
 54%|█████▍    | 406/750 [44:29<49:34,  8.65s/it]
                                                 
{'loss': 0.1133, 'grad_norm': 0.09392616152763367, 'learning_rate': 0.00011900656106431562, 'epoch': 2.71}
+
 54%|█████▍    | 406/750 [44:29<49:34,  8.65s/it]
 54%|█████▍    | 407/750 [44:35<45:10,  7.90s/it]
                                                 
{'loss': 0.1079, 'grad_norm': 0.09541792422533035, 'learning_rate': 0.00011886027144409578, 'epoch': 2.71}
+
 54%|█████▍    | 407/750 [44:35<45:10,  7.90s/it]
 54%|█████▍    | 408/750 [44:41<42:11,  7.40s/it]
                                                 
{'loss': 0.1193, 'grad_norm': 0.10365637391805649, 'learning_rate': 0.00011871451998345427, 'epoch': 2.72}
+
 54%|█████▍    | 408/750 [44:41<42:11,  7.40s/it]
 55%|█████▍    | 409/750 [44:47<39:55,  7.02s/it]
                                                 
{'loss': 0.1081, 'grad_norm': 0.09888631850481033, 'learning_rate': 0.00011856930339089229, 'epoch': 2.73}
+
 55%|█████▍    | 409/750 [44:47<39:55,  7.02s/it]
 55%|█████▍    | 410/750 [44:53<38:38,  6.82s/it]
                                                 
{'loss': 0.1285, 'grad_norm': 0.09599032253026962, 'learning_rate': 0.00011842461840302649, 'epoch': 2.73}
+
 55%|█████▍    | 410/750 [44:53<38:38,  6.82s/it]
 55%|█████▍    | 411/750 [45:00<37:41,  6.67s/it]
                                                 
{'loss': 0.1027, 'grad_norm': 0.09334539622068405, 'learning_rate': 0.00011828046178428064, 'epoch': 2.74}
+
 55%|█████▍    | 411/750 [45:00<37:41,  6.67s/it]
 55%|█████▍    | 412/750 [45:06<36:47,  6.53s/it]
                                                 
{'loss': 0.1117, 'grad_norm': 0.09040253609418869, 'learning_rate': 0.00011813683032658212, 'epoch': 2.75}
+
 55%|█████▍    | 412/750 [45:06<36:47,  6.53s/it]
 55%|█████▌    | 413/750 [45:12<36:06,  6.43s/it]
                                                 
{'loss': 0.1089, 'grad_norm': 0.09478548169136047, 'learning_rate': 0.0001179937208490617, 'epoch': 2.75}
+
 55%|█████▌    | 413/750 [45:12<36:06,  6.43s/it]
 55%|█████▌    | 414/750 [45:18<35:37,  6.36s/it]
                                                 
{'loss': 0.1233, 'grad_norm': 0.09629514813423157, 'learning_rate': 0.00011785113019775794, 'epoch': 2.76}
+
 55%|█████▌    | 414/750 [45:18<35:37,  6.36s/it]
 55%|█████▌    | 415/750 [45:25<35:12,  6.31s/it]
                                                 
{'loss': 0.1238, 'grad_norm': 0.10360328108072281, 'learning_rate': 0.00011770905524532507, 'epoch': 2.77}
+
 55%|█████▌    | 415/750 [45:25<35:12,  6.31s/it]
 55%|█████▌    | 416/750 [45:31<34:58,  6.28s/it]
                                                 
{'loss': 0.1187, 'grad_norm': 0.0957934781908989, 'learning_rate': 0.00011756749289074503, 'epoch': 2.77}
+
 55%|█████▌    | 416/750 [45:31<34:58,  6.28s/it]
 56%|█████▌    | 417/750 [45:37<34:37,  6.24s/it]
                                                 
{'loss': 0.1042, 'grad_norm': 0.09110625833272934, 'learning_rate': 0.00011742644005904313, 'epoch': 2.78}
+
 56%|█████▌    | 417/750 [45:37<34:37,  6.24s/it]
 56%|█████▌    | 418/750 [45:43<34:19,  6.20s/it]
                                                 
{'loss': 0.1043, 'grad_norm': 0.09436056017875671, 'learning_rate': 0.00011728589370100743, 'epoch': 2.79}
+
 56%|█████▌    | 418/750 [45:43<34:19,  6.20s/it]
 56%|█████▌    | 419/750 [45:49<34:09,  6.19s/it]
                                                 
{'loss': 0.1161, 'grad_norm': 0.10103736817836761, 'learning_rate': 0.00011714585079291212, 'epoch': 2.79}
+
 56%|█████▌    | 419/750 [45:49<34:09,  6.19s/it]
 56%|█████▌    | 420/750 [45:55<34:07,  6.20s/it]
                                                 
{'loss': 0.1163, 'grad_norm': 0.09424997866153717, 'learning_rate': 0.00011700630833624395, 'epoch': 2.8}
+
 56%|█████▌    | 420/750 [45:55<34:07,  6.20s/it]
 56%|█████▌    | 421/750 [46:02<33:52,  6.18s/it]
                                                 
{'loss': 0.1151, 'grad_norm': 0.09667247533798218, 'learning_rate': 0.00011686726335743291, 'epoch': 2.81}
+
 56%|█████▌    | 421/750 [46:02<33:52,  6.18s/it]
 56%|█████▋    | 422/750 [46:08<33:51,  6.19s/it]
                                                 
{'loss': 0.124, 'grad_norm': 0.09493300318717957, 'learning_rate': 0.0001167287129075859, 'epoch': 2.81}
+
 56%|█████▋    | 422/750 [46:08<33:51,  6.19s/it]
 56%|█████▋    | 423/750 [46:14<33:45,  6.19s/it]
                                                 
{'loss': 0.1088, 'grad_norm': 0.0934746265411377, 'learning_rate': 0.00011659065406222409, 'epoch': 2.82}
+
 56%|█████▋    | 423/750 [46:14<33:45,  6.19s/it]
 57%|█████▋    | 424/750 [46:20<33:34,  6.18s/it]
                                                 
{'loss': 0.1026, 'grad_norm': 0.09114349633455276, 'learning_rate': 0.00011645308392102366, 'epoch': 2.83}
+
 57%|█████▋    | 424/750 [46:20<33:34,  6.18s/it]
 57%|█████▋    | 425/750 [46:26<33:21,  6.16s/it]
                                                 
{'loss': 0.1056, 'grad_norm': 0.09486070275306702, 'learning_rate': 0.00011631599960755992, 'epoch': 2.83}
+
 57%|█████▋    | 425/750 [46:26<33:21,  6.16s/it]
 57%|█████▋    | 426/750 [46:32<33:23,  6.18s/it]
                                                 
{'loss': 0.1101, 'grad_norm': 0.0891023576259613, 'learning_rate': 0.00011617939826905469, 'epoch': 2.84}
+
 57%|█████▋    | 426/750 [46:32<33:23,  6.18s/it]
 57%|█████▋    | 427/750 [46:39<33:18,  6.19s/it]
                                                 
{'loss': 0.1102, 'grad_norm': 0.09251202642917633, 'learning_rate': 0.00011604327707612684, 'epoch': 2.85}
+
 57%|█████▋    | 427/750 [46:39<33:18,  6.19s/it]
 57%|█████▋    | 428/750 [46:45<33:24,  6.22s/it]
                                                 
{'loss': 0.1089, 'grad_norm': 0.09286895394325256, 'learning_rate': 0.00011590763322254638, 'epoch': 2.85}
+
 57%|█████▋    | 428/750 [46:45<33:24,  6.22s/it]
 57%|█████▋    | 429/750 [46:51<32:58,  6.16s/it]
                                                 
{'loss': 0.1171, 'grad_norm': 0.09787318110466003, 'learning_rate': 0.00011577246392499127, 'epoch': 2.86}
+
 57%|█████▋    | 429/750 [46:51<32:58,  6.16s/it]
 57%|█████▋    | 430/750 [46:57<33:03,  6.20s/it]
                                                 
{'loss': 0.1059, 'grad_norm': 0.09293732047080994, 'learning_rate': 0.00011563776642280763, 'epoch': 2.87}
+
 57%|█████▋    | 430/750 [46:57<33:03,  6.20s/it]
 57%|█████▋    | 431/750 [47:03<32:55,  6.19s/it]
                                                 
{'loss': 0.1109, 'grad_norm': 0.09513161331415176, 'learning_rate': 0.00011550353797777291, 'epoch': 2.87}
+
 57%|█████▋    | 431/750 [47:03<32:55,  6.19s/it]
 58%|█████▊    | 432/750 [47:10<32:44,  6.18s/it]
                                                 
{'loss': 0.107, 'grad_norm': 0.09129812568426132, 'learning_rate': 0.00011536977587386216, 'epoch': 2.88}
+
 58%|█████▊    | 432/750 [47:10<32:44,  6.18s/it]
 58%|█████▊    | 433/750 [47:16<32:38,  6.18s/it]
                                                 
{'loss': 0.102, 'grad_norm': 0.09015312790870667, 'learning_rate': 0.00011523647741701704, 'epoch': 2.89}
+
 58%|█████▊    | 433/750 [47:16<32:38,  6.18s/it]
 58%|█████▊    | 434/750 [47:22<32:32,  6.18s/it]
                                                 
{'loss': 0.1203, 'grad_norm': 0.09451872855424881, 'learning_rate': 0.00011510363993491814, 'epoch': 2.89}
+
 58%|█████▊    | 434/750 [47:22<32:32,  6.18s/it]
 58%|█████▊    | 435/750 [47:28<32:25,  6.18s/it]
                                                 
{'loss': 0.1182, 'grad_norm': 0.09999062120914459, 'learning_rate': 0.00011497126077675976, 'epoch': 2.9}
+
 58%|█████▊    | 435/750 [47:28<32:25,  6.18s/it]
 58%|█████▊    | 436/750 [47:34<32:17,  6.17s/it]
                                                 
{'loss': 0.127, 'grad_norm': 0.09826690703630447, 'learning_rate': 0.00011483933731302795, 'epoch': 2.91}
+
 58%|█████▊    | 436/750 [47:34<32:17,  6.17s/it]
 58%|█████▊    | 437/750 [47:40<32:03,  6.15s/it]
                                                 
{'loss': 0.1062, 'grad_norm': 0.0979766994714737, 'learning_rate': 0.00011470786693528087, 'epoch': 2.91}
+
 58%|█████▊    | 437/750 [47:40<32:03,  6.15s/it]
 58%|█████▊    | 438/750 [47:46<31:51,  6.13s/it]
                                                 
{'loss': 0.1277, 'grad_norm': 0.10701143741607666, 'learning_rate': 0.00011457684705593247, 'epoch': 2.92}
+
 58%|█████▊    | 438/750 [47:46<31:51,  6.13s/it]
 59%|█████▊    | 439/750 [47:53<31:47,  6.13s/it]
                                                 
{'loss': 0.1221, 'grad_norm': 0.10225667804479599, 'learning_rate': 0.00011444627510803829, 'epoch': 2.93}
+
 59%|█████▊    | 439/750 [47:53<31:47,  6.13s/it]
 59%|█████▊    | 440/750 [47:59<31:42,  6.14s/it]
                                                 
{'loss': 0.1081, 'grad_norm': 0.09214270859956741, 'learning_rate': 0.00011431614854508447, 'epoch': 2.93}
+
 59%|█████▊    | 440/750 [47:59<31:42,  6.14s/it]
 59%|█████▉    | 441/750 [48:05<31:38,  6.14s/it]
                                                 
{'loss': 0.0958, 'grad_norm': 0.09254073351621628, 'learning_rate': 0.00011418646484077904, 'epoch': 2.94}
+
 59%|█████▉    | 441/750 [48:05<31:38,  6.14s/it]
 59%|█████▉    | 442/750 [48:11<31:28,  6.13s/it]
                                                 
{'loss': 0.1081, 'grad_norm': 0.09329082071781158, 'learning_rate': 0.00011405722148884592, 'epoch': 2.95}
+
 59%|█████▉    | 442/750 [48:11<31:28,  6.13s/it]
 59%|█████▉    | 443/750 [48:17<31:25,  6.14s/it]
                                                 
{'loss': 0.1118, 'grad_norm': 0.0971725657582283, 'learning_rate': 0.00011392841600282156, 'epoch': 2.95}
+
 59%|█████▉    | 443/750 [48:17<31:25,  6.14s/it]
 59%|█████▉    | 444/750 [48:23<31:16,  6.13s/it]
                                                 
{'loss': 0.1331, 'grad_norm': 0.09979835152626038, 'learning_rate': 0.00011380004591585387, 'epoch': 2.96}
+
 59%|█████▉    | 444/750 [48:23<31:16,  6.13s/it]
 59%|█████▉    | 445/750 [48:29<31:07,  6.12s/it]
                                                 
{'loss': 0.1158, 'grad_norm': 0.09554321318864822, 'learning_rate': 0.00011367210878050402, 'epoch': 2.97}
+
 59%|█████▉    | 445/750 [48:29<31:07,  6.12s/it]
 59%|█████▉    | 446/750 [48:35<30:57,  6.11s/it]
                                                 
{'loss': 0.1127, 'grad_norm': 0.09346331655979156, 'learning_rate': 0.00011354460216855028, 'epoch': 2.97}
+
 59%|█████▉    | 446/750 [48:35<30:57,  6.11s/it]
 60%|█████▉    | 447/750 [48:42<30:56,  6.13s/it]
                                                 
{'loss': 0.1203, 'grad_norm': 0.09465193003416061, 'learning_rate': 0.00011341752367079442, 'epoch': 2.98}
+
 60%|█████▉    | 447/750 [48:42<30:56,  6.13s/it]
 60%|█████▉    | 448/750 [48:48<31:07,  6.19s/it]
                                                 
{'loss': 0.1161, 'grad_norm': 0.09860458970069885, 'learning_rate': 0.00011329087089687071, 'epoch': 2.99}
+
 60%|█████▉    | 448/750 [48:48<31:07,  6.19s/it]
 60%|█████▉    | 449/750 [48:54<31:01,  6.18s/it]
                                                 
{'loss': 0.1152, 'grad_norm': 0.0986085906624794, 'learning_rate': 0.00011316464147505693, 'epoch': 2.99}
+
 60%|█████▉    | 449/750 [48:54<31:01,  6.18s/it]
 60%|██████    | 450/750 [49:02<33:52,  6.77s/it]
                                                 
{'loss': 0.1113, 'grad_norm': 0.09433763474225998, 'learning_rate': 0.0001130388330520878, 'epoch': 3.0}
+
 60%|██████    | 450/750 [49:02<33:52,  6.77s/it]
 60%|██████    | 451/750 [49:17<45:17,  9.09s/it]
                                                 
{'loss': 0.1027, 'grad_norm': 0.0979873463511467, 'learning_rate': 0.00011291344329297086, 'epoch': 3.01}
+
 60%|██████    | 451/750 [49:17<45:17,  9.09s/it]
 60%|██████    | 452/750 [49:23<40:47,  8.21s/it]
                                                 
{'loss': 0.0972, 'grad_norm': 0.08797840774059296, 'learning_rate': 0.0001127884698808043, 'epoch': 3.01}
+
 60%|██████    | 452/750 [49:23<40:47,  8.21s/it]
 60%|██████    | 453/750 [49:29<37:31,  7.58s/it]
                                                 
{'loss': 0.0863, 'grad_norm': 0.09042159467935562, 'learning_rate': 0.00011266391051659725, 'epoch': 3.02}
+
 60%|██████    | 453/750 [49:29<37:31,  7.58s/it]
 61%|██████    | 454/750 [49:35<35:19,  7.16s/it]
                                                 
{'loss': 0.0795, 'grad_norm': 0.08782897889614105, 'learning_rate': 0.00011253976291909212, 'epoch': 3.03}
+
 61%|██████    | 454/750 [49:35<35:19,  7.16s/it]
 61%|██████    | 455/750 [49:41<33:43,  6.86s/it]
                                                 
{'loss': 0.0955, 'grad_norm': 0.10429650545120239, 'learning_rate': 0.00011241602482458911, 'epoch': 3.03}
+
 61%|██████    | 455/750 [49:41<33:43,  6.86s/it]
 61%|██████    | 456/750 [49:47<32:28,  6.63s/it]
                                                 
{'loss': 0.0826, 'grad_norm': 0.10772597789764404, 'learning_rate': 0.00011229269398677288, 'epoch': 3.04}
+
 61%|██████    | 456/750 [49:47<32:28,  6.63s/it]
 61%|██████    | 457/750 [49:54<31:43,  6.50s/it]
                                                 
{'loss': 0.0887, 'grad_norm': 0.12117308378219604, 'learning_rate': 0.00011216976817654119, 'epoch': 3.05}
+
 61%|██████    | 457/750 [49:54<31:43,  6.50s/it]
 61%|██████    | 458/750 [50:00<31:06,  6.39s/it]
                                                 
{'loss': 0.0944, 'grad_norm': 0.12423322349786758, 'learning_rate': 0.00011204724518183582, 'epoch': 3.05}
+
 61%|██████    | 458/750 [50:00<31:06,  6.39s/it]
 61%|██████    | 459/750 [50:06<30:44,  6.34s/it]
                                                 
{'loss': 0.0919, 'grad_norm': 0.11315707117319107, 'learning_rate': 0.00011192512280747522, 'epoch': 3.06}
+
 61%|██████    | 459/750 [50:06<30:44,  6.34s/it]
 61%|██████▏   | 460/750 [50:12<30:31,  6.31s/it]
                                                 
{'loss': 0.1039, 'grad_norm': 0.10856442153453827, 'learning_rate': 0.00011180339887498949, 'epoch': 3.07}
+
 61%|██████▏   | 460/750 [50:12<30:31,  6.31s/it]
 61%|██████▏   | 461/750 [50:18<30:09,  6.26s/it]
                                                 
{'loss': 0.0963, 'grad_norm': 0.10252365469932556, 'learning_rate': 0.000111682071222457, 'epoch': 3.07}
+
 61%|██████▏   | 461/750 [50:18<30:09,  6.26s/it]
 62%|██████▏   | 462/750 [50:25<29:49,  6.21s/it]
                                                 
{'loss': 0.0996, 'grad_norm': 0.11418426036834717, 'learning_rate': 0.00011156113770434329, 'epoch': 3.08}
+
 62%|██████▏   | 462/750 [50:25<29:49,  6.21s/it]
 62%|██████▏   | 463/750 [50:31<29:44,  6.22s/it]
                                                 
{'loss': 0.0909, 'grad_norm': 0.10253693908452988, 'learning_rate': 0.00011144059619134151, 'epoch': 3.09}
+
 62%|██████▏   | 463/750 [50:31<29:44,  6.22s/it]
 62%|██████▏   | 464/750 [50:37<29:32,  6.20s/it]
                                                 
{'loss': 0.0872, 'grad_norm': 0.1007380560040474, 'learning_rate': 0.00011132044457021517, 'epoch': 3.09}
+
 62%|██████▏   | 464/750 [50:37<29:32,  6.20s/it]
 62%|██████▏   | 465/750 [50:43<29:16,  6.16s/it]
                                                 
{'loss': 0.093, 'grad_norm': 0.10790285468101501, 'learning_rate': 0.00011120068074364232, 'epoch': 3.1}
+
 62%|██████▏   | 465/750 [50:43<29:16,  6.16s/it]
 62%|██████▏   | 466/750 [50:49<29:18,  6.19s/it]
                                                 
{'loss': 0.0846, 'grad_norm': 0.10167799890041351, 'learning_rate': 0.000111081302630062, 'epoch': 3.11}
+
 62%|██████▏   | 466/750 [50:49<29:18,  6.19s/it]
 62%|██████▏   | 467/750 [50:55<29:09,  6.18s/it]
                                                 
{'loss': 0.0883, 'grad_norm': 0.10658925026655197, 'learning_rate': 0.0001109623081635221, 'epoch': 3.11}
+
 62%|██████▏   | 467/750 [50:55<29:09,  6.18s/it]
 62%|██████▏   | 468/750 [51:02<29:13,  6.22s/it]
                                                 
{'loss': 0.1005, 'grad_norm': 0.11666425317525864, 'learning_rate': 0.00011084369529352937, 'epoch': 3.12}
+
 62%|██████▏   | 468/750 [51:02<29:13,  6.22s/it]
 63%|██████▎   | 469/750 [51:08<28:55,  6.17s/it]
                                                 
{'loss': 0.0905, 'grad_norm': 0.1135924905538559, 'learning_rate': 0.00011072546198490086, 'epoch': 3.13}
+
 63%|██████▎   | 469/750 [51:08<28:55,  6.17s/it]
 63%|██████▎   | 470/750 [51:14<28:44,  6.16s/it]
                                                 
{'loss': 0.0843, 'grad_norm': 0.12314590066671371, 'learning_rate': 0.00011060760621761728, 'epoch': 3.13}
+
 63%|██████▎   | 470/750 [51:14<28:44,  6.16s/it]
 63%|██████▎   | 471/750 [51:20<28:36,  6.15s/it]
                                                 
{'loss': 0.103, 'grad_norm': 0.11246594041585922, 'learning_rate': 0.00011049012598667817, 'epoch': 3.14}
+
 63%|██████▎   | 471/750 [51:20<28:36,  6.15s/it]
 63%|██████▎   | 472/750 [51:26<28:25,  6.14s/it]
                                                 
{'loss': 0.092, 'grad_norm': 0.11147676408290863, 'learning_rate': 0.00011037301930195852, 'epoch': 3.15}
+
 63%|██████▎   | 472/750 [51:26<28:25,  6.14s/it]
 63%|██████▎   | 473/750 [51:32<28:19,  6.14s/it]
                                                 
{'loss': 0.0861, 'grad_norm': 0.1144125834107399, 'learning_rate': 0.00011025628418806717, 'epoch': 3.15}
+
 63%|██████▎   | 473/750 [51:32<28:19,  6.14s/it]
 63%|██████▎   | 474/750 [51:38<28:13,  6.13s/it]
                                                 
{'loss': 0.0985, 'grad_norm': 0.10731994360685349, 'learning_rate': 0.00011013991868420692, 'epoch': 3.16}
+
 63%|██████▎   | 474/750 [51:38<28:13,  6.13s/it]
 63%|██████▎   | 475/750 [51:45<28:06,  6.13s/it]
                                                 
{'loss': 0.086, 'grad_norm': 0.1045454889535904, 'learning_rate': 0.00011002392084403617, 'epoch': 3.17}
+
 63%|██████▎   | 475/750 [51:45<28:06,  6.13s/it]
 63%|██████▎   | 476/750 [51:51<28:00,  6.13s/it]
                                                 
{'loss': 0.0972, 'grad_norm': 0.11227450519800186, 'learning_rate': 0.00010990828873553218, 'epoch': 3.17}
+
 63%|██████▎   | 476/750 [51:51<28:00,  6.13s/it]
 64%|██████▎   | 477/750 [51:57<27:48,  6.11s/it]
                                                 
{'loss': 0.0988, 'grad_norm': 0.114931121468544, 'learning_rate': 0.00010979302044085591, 'epoch': 3.18}
+
 64%|██████▎   | 477/750 [51:57<27:48,  6.11s/it]
 64%|██████▎   | 478/750 [52:03<27:38,  6.10s/it]
                                                 
{'loss': 0.0896, 'grad_norm': 0.11316587775945663, 'learning_rate': 0.00010967811405621847, 'epoch': 3.19}
+
 64%|██████▎   | 478/750 [52:03<27:38,  6.10s/it]
 64%|██████▍   | 479/750 [52:09<27:36,  6.11s/it]
                                                 
{'loss': 0.0886, 'grad_norm': 0.10645253211259842, 'learning_rate': 0.000109563567691749, 'epoch': 3.19}
+
 64%|██████▍   | 479/750 [52:09<27:36,  6.11s/it]
 64%|██████▍   | 480/750 [52:15<27:35,  6.13s/it]
                                                 
{'loss': 0.0975, 'grad_norm': 0.11300535500049591, 'learning_rate': 0.00010944937947136412, 'epoch': 3.2}
+
 64%|██████▍   | 480/750 [52:15<27:35,  6.13s/it]
 64%|██████▍   | 481/750 [52:21<27:32,  6.14s/it]
                                                 
{'loss': 0.0833, 'grad_norm': 0.10558468848466873, 'learning_rate': 0.00010933554753263895, 'epoch': 3.21}
+
 64%|██████▍   | 481/750 [52:21<27:32,  6.14s/it]
 64%|██████▍   | 482/750 [52:27<27:22,  6.13s/it]
                                                 
{'loss': 0.0894, 'grad_norm': 0.12372148782014847, 'learning_rate': 0.00010922207002667943, 'epoch': 3.21}
+
 64%|██████▍   | 482/750 [52:27<27:22,  6.13s/it]
 64%|██████▍   | 483/750 [52:33<27:13,  6.12s/it]
                                                 
{'loss': 0.0887, 'grad_norm': 0.12184583395719528, 'learning_rate': 0.0001091089451179962, 'epoch': 3.22}
+
 64%|██████▍   | 483/750 [52:33<27:13,  6.12s/it]
 65%|██████▍   | 484/750 [52:40<27:10,  6.13s/it]
                                                 
{'loss': 0.0974, 'grad_norm': 0.13584524393081665, 'learning_rate': 0.00010899617098437999, 'epoch': 3.23}
+
 65%|██████▍   | 484/750 [52:40<27:10,  6.13s/it]
 65%|██████▍   | 485/750 [52:46<27:24,  6.21s/it]
                                                 
{'loss': 0.0969, 'grad_norm': 0.11636198312044144, 'learning_rate': 0.00010888374581677828, 'epoch': 3.23}
+
 65%|██████▍   | 485/750 [52:46<27:24,  6.21s/it]
 65%|██████▍   | 486/750 [52:52<27:14,  6.19s/it]
                                                 
{'loss': 0.092, 'grad_norm': 0.10711462795734406, 'learning_rate': 0.00010877166781917343, 'epoch': 3.24}
+
 65%|██████▍   | 486/750 [52:52<27:14,  6.19s/it]
 65%|██████▍   | 487/750 [52:58<27:14,  6.22s/it]
                                                 
{'loss': 0.0996, 'grad_norm': 0.1141488328576088, 'learning_rate': 0.00010865993520846211, 'epoch': 3.25}
+
 65%|██████▍   | 487/750 [52:58<27:14,  6.22s/it]
 65%|██████▌   | 488/750 [53:05<27:03,  6.20s/it]
                                                 
{'loss': 0.0965, 'grad_norm': 0.10600733011960983, 'learning_rate': 0.0001085485462143363, 'epoch': 3.25}
+
 65%|██████▌   | 488/750 [53:05<27:03,  6.20s/it]
 65%|██████▌   | 489/750 [53:11<26:46,  6.16s/it]
                                                 
{'loss': 0.088, 'grad_norm': 0.1105402484536171, 'learning_rate': 0.00010843749907916526, 'epoch': 3.26}
+
 65%|██████▌   | 489/750 [53:11<26:46,  6.16s/it]
 65%|██████▌   | 490/750 [53:17<26:46,  6.18s/it]
                                                 
{'loss': 0.0968, 'grad_norm': 0.11513582617044449, 'learning_rate': 0.0001083267920578793, 'epoch': 3.27}
+
 65%|██████▌   | 490/750 [53:17<26:46,  6.18s/it]
 65%|██████▌   | 491/750 [53:23<26:36,  6.16s/it]
                                                 
{'loss': 0.0929, 'grad_norm': 0.1109117865562439, 'learning_rate': 0.00010821642341785439, 'epoch': 3.27}
+
 65%|██████▌   | 491/750 [53:23<26:36,  6.16s/it]
 66%|██████▌   | 492/750 [53:29<26:25,  6.15s/it]
                                                 
{'loss': 0.1013, 'grad_norm': 0.11626242101192474, 'learning_rate': 0.00010810639143879838, 'epoch': 3.28}
+
 66%|██████▌   | 492/750 [53:29<26:25,  6.15s/it]
 66%|██████▌   | 493/750 [53:35<26:14,  6.13s/it]
                                                 
{'loss': 0.0955, 'grad_norm': 0.11476726830005646, 'learning_rate': 0.00010799669441263838, 'epoch': 3.29}
+
 66%|██████▌   | 493/750 [53:35<26:14,  6.13s/it]
 66%|██████▌   | 494/750 [53:41<26:12,  6.14s/it]
                                                 
{'loss': 0.0914, 'grad_norm': 0.11616263538599014, 'learning_rate': 0.0001078873306434093, 'epoch': 3.29}
+
 66%|██████▌   | 494/750 [53:41<26:12,  6.14s/it]
 66%|██████▌   | 495/750 [53:48<26:04,  6.13s/it]
                                                 
{'loss': 0.1006, 'grad_norm': 0.12230973690748215, 'learning_rate': 0.00010777829844714388, 'epoch': 3.3}
+
 66%|██████▌   | 495/750 [53:48<26:04,  6.13s/it]
 66%|██████▌   | 496/750 [53:54<25:58,  6.13s/it]
                                                 
{'loss': 0.0892, 'grad_norm': 0.11710097640752792, 'learning_rate': 0.0001076695961517636, 'epoch': 3.31}
+
 66%|██████▌   | 496/750 [53:54<25:58,  6.13s/it]
 66%|██████▋   | 497/750 [54:00<25:49,  6.12s/it]
                                                 
{'loss': 0.0969, 'grad_norm': 0.11430113017559052, 'learning_rate': 0.0001075612220969712, 'epoch': 3.31}
+
 66%|██████▋   | 497/750 [54:00<25:49,  6.12s/it]
 66%|██████▋   | 498/750 [54:06<25:44,  6.13s/it]
                                                 
{'loss': 0.0888, 'grad_norm': 0.11162873357534409, 'learning_rate': 0.00010745317463414389, 'epoch': 3.32}
+
 66%|██████▋   | 498/750 [54:06<25:44,  6.13s/it]
 67%|██████▋   | 499/750 [54:12<25:41,  6.14s/it]
                                                 
{'loss': 0.0873, 'grad_norm': 0.10880569368600845, 'learning_rate': 0.00010734545212622828, 'epoch': 3.33}
+
 67%|██████▋   | 499/750 [54:12<25:41,  6.14s/it]
 67%|██████▋   | 500/750 [54:18<25:36,  6.15s/it]
                                                 
{'loss': 0.1134, 'grad_norm': 0.13140757381916046, 'learning_rate': 0.00010723805294763608, 'epoch': 3.33}
+
 67%|██████▋   | 500/750 [54:18<25:36,  6.15s/it]
 67%|██████▋   | 501/750 [54:24<25:27,  6.13s/it]
                                                 
{'loss': 0.1074, 'grad_norm': 0.11947689205408096, 'learning_rate': 0.00010713097548414107, 'epoch': 3.34}
+
 67%|██████▋   | 501/750 [54:24<25:27,  6.13s/it]
 67%|██████▋   | 502/750 [54:30<25:23,  6.14s/it]
                                                 
{'loss': 0.0945, 'grad_norm': 0.11526649445295334, 'learning_rate': 0.00010702421813277718, 'epoch': 3.35}
+
 67%|██████▋   | 502/750 [54:30<25:23,  6.14s/it]
 67%|██████▋   | 503/750 [54:37<25:26,  6.18s/it]
                                                 
{'loss': 0.1011, 'grad_norm': 0.115763358771801, 'learning_rate': 0.0001069177793017378, 'epoch': 3.35}
+
 67%|██████▋   | 503/750 [54:37<25:26,  6.18s/it]
 67%|██████▋   | 504/750 [54:43<25:14,  6.16s/it]
                                                 
{'loss': 0.1008, 'grad_norm': 0.1159386858344078, 'learning_rate': 0.00010681165741027595, 'epoch': 3.36}
+
 67%|██████▋   | 504/750 [54:43<25:14,  6.16s/it]
 67%|██████▋   | 505/750 [54:49<25:06,  6.15s/it]
                                                 
{'loss': 0.0933, 'grad_norm': 0.11035444587469101, 'learning_rate': 0.00010670585088860586, 'epoch': 3.37}
+
 67%|██████▋   | 505/750 [54:49<25:06,  6.15s/it]
 67%|██████▋   | 506/750 [54:55<25:10,  6.19s/it]
                                                 
{'loss': 0.1048, 'grad_norm': 0.11857625842094421, 'learning_rate': 0.00010660035817780521, 'epoch': 3.37}
+
 67%|██████▋   | 506/750 [54:55<25:10,  6.19s/it]
 68%|██████▊   | 507/750 [55:01<25:03,  6.19s/it]
                                                 
{'loss': 0.0891, 'grad_norm': 0.11901260167360306, 'learning_rate': 0.00010649517772971891, 'epoch': 3.38}
+
 68%|██████▊   | 507/750 [55:01<25:03,  6.19s/it]
 68%|██████▊   | 508/750 [55:08<24:48,  6.15s/it]
                                                 
{'loss': 0.0785, 'grad_norm': 0.11297748982906342, 'learning_rate': 0.00010639030800686347, 'epoch': 3.39}
+
 68%|██████▊   | 508/750 [55:08<24:48,  6.15s/it]
 68%|██████▊   | 509/750 [55:14<24:43,  6.15s/it]
                                                 
{'loss': 0.0819, 'grad_norm': 0.10936430841684341, 'learning_rate': 0.0001062857474823327, 'epoch': 3.39}
+
 68%|██████▊   | 509/750 [55:14<24:43,  6.15s/it]
 68%|██████▊   | 510/750 [55:20<24:36,  6.15s/it]
                                                 
{'loss': 0.0947, 'grad_norm': 0.11804181337356567, 'learning_rate': 0.00010618149463970438, 'epoch': 3.4}
+
 68%|██████▊   | 510/750 [55:20<24:36,  6.15s/it]
 68%|██████▊   | 511/750 [55:26<24:29,  6.15s/it]
                                                 
{'loss': 0.0914, 'grad_norm': 0.11075980961322784, 'learning_rate': 0.00010607754797294777, 'epoch': 3.41}
+
 68%|██████▊   | 511/750 [55:26<24:29,  6.15s/it]
 68%|██████▊   | 512/750 [55:32<24:21,  6.14s/it]
                                                 
{'loss': 0.1, 'grad_norm': 0.11250878870487213, 'learning_rate': 0.00010597390598633231, 'epoch': 3.41}
+
 68%|██████▊   | 512/750 [55:32<24:21,  6.14s/it]
 68%|██████▊   | 513/750 [55:38<24:13,  6.13s/it]
                                                 
{'loss': 0.0948, 'grad_norm': 0.137189581990242, 'learning_rate': 0.00010587056719433728, 'epoch': 3.42}
+
 68%|██████▊   | 513/750 [55:38<24:13,  6.13s/it]
 69%|██████▊   | 514/750 [55:44<24:05,  6.12s/it]
                                                 
{'loss': 0.0901, 'grad_norm': 0.11442418396472931, 'learning_rate': 0.00010576753012156217, 'epoch': 3.43}
+
 69%|██████▊   | 514/750 [55:44<24:05,  6.12s/it]
 69%|██████▊   | 515/750 [55:50<24:00,  6.13s/it]
                                                 
{'loss': 0.1015, 'grad_norm': 0.12193969637155533, 'learning_rate': 0.0001056647933026385, 'epoch': 3.43}
+
 69%|██████▊   | 515/750 [55:50<24:00,  6.13s/it]
 69%|██████▉   | 516/750 [55:57<23:55,  6.14s/it]
                                                 
{'loss': 0.1069, 'grad_norm': 0.11846344918012619, 'learning_rate': 0.00010556235528214203, 'epoch': 3.44}
+
 69%|██████▉   | 516/750 [55:57<23:55,  6.14s/it]
 69%|██████▉   | 517/750 [56:03<23:48,  6.13s/it]
                                                 
{'loss': 0.0923, 'grad_norm': 0.11032267659902573, 'learning_rate': 0.00010546021461450624, 'epoch': 3.45}
+
 69%|██████▉   | 517/750 [56:03<23:48,  6.13s/it]
 69%|██████▉   | 518/750 [56:09<23:44,  6.14s/it]
                                                 
{'loss': 0.0951, 'grad_norm': 0.11216137558221817, 'learning_rate': 0.00010535836986393678, 'epoch': 3.45}
+
 69%|██████▉   | 518/750 [56:09<23:44,  6.14s/it]
 69%|██████▉   | 519/750 [56:15<23:34,  6.12s/it]
                                                 
{'loss': 0.1094, 'grad_norm': 0.12297376990318298, 'learning_rate': 0.00010525681960432646, 'epoch': 3.46}
+
 69%|██████▉   | 519/750 [56:15<23:34,  6.12s/it]
 69%|██████▉   | 520/750 [56:21<23:31,  6.14s/it]
                                                 
{'loss': 0.085, 'grad_norm': 0.11285136640071869, 'learning_rate': 0.00010515556241917166, 'epoch': 3.47}
+
 69%|██████▉   | 520/750 [56:21<23:31,  6.14s/it]
 69%|██████▉   | 521/750 [56:27<23:21,  6.12s/it]
                                                 
{'loss': 0.1032, 'grad_norm': 0.11205510795116425, 'learning_rate': 0.00010505459690148908, 'epoch': 3.47}
+
 69%|██████▉   | 521/750 [56:27<23:21,  6.12s/it]
 70%|██████▉   | 522/750 [56:33<23:15,  6.12s/it]
                                                 
{'loss': 0.097, 'grad_norm': 0.11545735597610474, 'learning_rate': 0.00010495392165373376, 'epoch': 3.48}
+
 70%|██████▉   | 522/750 [56:33<23:15,  6.12s/it]
 70%|██████▉   | 523/750 [56:40<23:16,  6.15s/it]
                                                 
{'loss': 0.0938, 'grad_norm': 0.12078775465488434, 'learning_rate': 0.00010485353528771787, 'epoch': 3.49}
+
 70%|██████▉   | 523/750 [56:40<23:16,  6.15s/it]
 70%|██████▉   | 524/750 [56:46<23:07,  6.14s/it]
                                                 
{'loss': 0.0878, 'grad_norm': 0.1146346852183342, 'learning_rate': 0.00010475343642453021, 'epoch': 3.49}
+
 70%|██████▉   | 524/750 [56:46<23:07,  6.14s/it]
 70%|███████   | 525/750 [56:52<23:07,  6.16s/it]
                                                 
{'loss': 0.091, 'grad_norm': 0.11764247715473175, 'learning_rate': 0.00010465362369445672, 'epoch': 3.5}
+
 70%|███████   | 525/750 [56:52<23:07,  6.16s/it]
 70%|███████   | 526/750 [56:58<23:02,  6.17s/it]
                                                 
{'loss': 0.0951, 'grad_norm': 0.1143137738108635, 'learning_rate': 0.00010455409573690175, 'epoch': 3.51}
+
 70%|███████   | 526/750 [56:58<23:02,  6.17s/it]
 70%|███████   | 527/750 [57:04<22:57,  6.18s/it]
                                                 
{'loss': 0.0993, 'grad_norm': 0.12190832942724228, 'learning_rate': 0.00010445485120031016, 'epoch': 3.51}
+
 70%|███████   | 527/750 [57:04<22:57,  6.18s/it]
 70%|███████   | 528/750 [57:10<22:46,  6.15s/it]
                                                 
{'loss': 0.0853, 'grad_norm': 0.11202817410230637, 'learning_rate': 0.00010435588874209024, 'epoch': 3.52}
+
 70%|███████   | 528/750 [57:10<22:46,  6.15s/it]
 71%|███████   | 529/750 [57:16<22:37,  6.14s/it]
                                                 
{'loss': 0.0954, 'grad_norm': 0.11697743833065033, 'learning_rate': 0.00010425720702853738, 'epoch': 3.53}
+
 71%|███████   | 529/750 [57:17<22:37,  6.14s/it]
 71%|███████   | 530/750 [57:23<22:33,  6.15s/it]
                                                 
{'loss': 0.1066, 'grad_norm': 0.12133266776800156, 'learning_rate': 0.00010415880473475868, 'epoch': 3.53}
+
 71%|███████   | 530/750 [57:23<22:33,  6.15s/it]
 71%|███████   | 531/750 [57:29<22:26,  6.15s/it]
                                                 
{'loss': 0.0975, 'grad_norm': 0.11761889606714249, 'learning_rate': 0.0001040606805445981, 'epoch': 3.54}
+
 71%|███████   | 531/750 [57:29<22:26,  6.15s/it]
 71%|███████   | 532/750 [57:35<22:17,  6.14s/it]
                                                 
{'loss': 0.0952, 'grad_norm': 0.11622606962919235, 'learning_rate': 0.00010396283315056266, 'epoch': 3.55}
+
 71%|███████   | 532/750 [57:35<22:17,  6.14s/it]
 71%|███████   | 533/750 [57:41<22:12,  6.14s/it]
                                                 
{'loss': 0.0897, 'grad_norm': 0.10772895812988281, 'learning_rate': 0.00010386526125374917, 'epoch': 3.55}
+
 71%|███████   | 533/750 [57:41<22:12,  6.14s/it]
 71%|███████   | 534/750 [57:47<22:04,  6.13s/it]
                                                 
{'loss': 0.1012, 'grad_norm': 0.12620706856250763, 'learning_rate': 0.00010376796356377186, 'epoch': 3.56}
+
 71%|███████   | 534/750 [57:47<22:04,  6.13s/it]
 71%|███████▏  | 535/750 [57:53<22:00,  6.14s/it]
                                                 
{'loss': 0.1013, 'grad_norm': 0.12111026793718338, 'learning_rate': 0.00010367093879869068, 'epoch': 3.57}
+
 71%|███████▏  | 535/750 [57:53<22:00,  6.14s/it]
 71%|███████▏  | 536/750 [57:59<21:54,  6.14s/it]
                                                 
{'loss': 0.0943, 'grad_norm': 0.1202906146645546, 'learning_rate': 0.00010357418568494037, 'epoch': 3.57}
+
 71%|███████▏  | 536/750 [57:59<21:54,  6.14s/it]
 72%|███████▏  | 537/750 [58:06<21:45,  6.13s/it]
                                                 
{'loss': 0.091, 'grad_norm': 0.12163220345973969, 'learning_rate': 0.00010347770295726023, 'epoch': 3.58}
+
 72%|███████▏  | 537/750 [58:06<21:45,  6.13s/it]
 72%|███████▏  | 538/750 [58:12<21:40,  6.14s/it]
                                                 
{'loss': 0.1032, 'grad_norm': 0.12050784379243851, 'learning_rate': 0.00010338148935862473, 'epoch': 3.59}
+
 72%|███████▏  | 538/750 [58:12<21:40,  6.14s/it]
 72%|███████▏  | 539/750 [58:18<21:36,  6.14s/it]
                                                 
{'loss': 0.0972, 'grad_norm': 0.11387603729963303, 'learning_rate': 0.00010328554364017445, 'epoch': 3.59}
+
 72%|███████▏  | 539/750 [58:18<21:36,  6.14s/it]
 72%|███████▏  | 540/750 [58:24<21:34,  6.16s/it]
                                                 
{'loss': 0.0965, 'grad_norm': 0.12409260869026184, 'learning_rate': 0.00010318986456114837, 'epoch': 3.6}
+
 72%|███████▏  | 540/750 [58:24<21:34,  6.16s/it]
 72%|███████▏  | 541/750 [58:30<21:38,  6.21s/it]
                                                 
{'loss': 0.096, 'grad_norm': 0.1193445697426796, 'learning_rate': 0.00010309445088881625, 'epoch': 3.61}
+
 72%|███████▏  | 541/750 [58:30<21:38,  6.21s/it]
 72%|███████▏  | 542/750 [58:37<21:25,  6.18s/it]
                                                 
{'loss': 0.0993, 'grad_norm': 0.12432117760181427, 'learning_rate': 0.00010299930139841185, 'epoch': 3.61}
+
 72%|███████▏  | 542/750 [58:37<21:25,  6.18s/it]
 72%|███████▏  | 543/750 [58:43<21:22,  6.20s/it]
                                                 
{'loss': 0.0952, 'grad_norm': 0.11916223913431168, 'learning_rate': 0.00010290441487306716, 'epoch': 3.62}
+
 72%|███████▏  | 543/750 [58:43<21:22,  6.20s/it]
 73%|███████▎  | 544/750 [58:49<21:13,  6.18s/it]
                                                 
{'loss': 0.0897, 'grad_norm': 0.11477713286876678, 'learning_rate': 0.00010280979010374679, 'epoch': 3.63}
+
 73%|███████▎  | 544/750 [58:49<21:13,  6.18s/it]
 73%|███████▎  | 545/750 [58:55<21:02,  6.16s/it]
                                                 
{'loss': 0.0883, 'grad_norm': 0.12097438424825668, 'learning_rate': 0.00010271542588918341, 'epoch': 3.63}
+
 73%|███████▎  | 545/750 [58:55<21:02,  6.16s/it]
 73%|███████▎  | 546/750 [59:01<20:59,  6.18s/it]
                                                 
{'loss': 0.0879, 'grad_norm': 0.11194033175706863, 'learning_rate': 0.00010262132103581367, 'epoch': 3.64}
+
 73%|███████▎  | 546/750 [59:01<20:59,  6.18s/it]
 73%|███████▎  | 547/750 [59:07<20:50,  6.16s/it]
                                                 
{'loss': 0.0946, 'grad_norm': 0.1151815876364708, 'learning_rate': 0.0001025274743577147, 'epoch': 3.65}
+
 73%|███████▎  | 547/750 [59:07<20:50,  6.16s/it]
 73%|███████▎  | 548/750 [59:14<20:44,  6.16s/it]
                                                 
{'loss': 0.0973, 'grad_norm': 0.12100318819284439, 'learning_rate': 0.00010243388467654152, 'epoch': 3.65}
+
 73%|███████▎  | 548/750 [59:14<20:44,  6.16s/it]
 73%|███████▎  | 549/750 [59:20<20:36,  6.15s/it]
                                                 
{'loss': 0.1028, 'grad_norm': 0.1201552301645279, 'learning_rate': 0.0001023405508214647, 'epoch': 3.66}
+
 73%|███████▎  | 549/750 [59:20<20:36,  6.15s/it]
 73%|███████▎  | 550/750 [59:26<20:24,  6.12s/it]
                                                 
{'loss': 0.1126, 'grad_norm': 0.12132446467876434, 'learning_rate': 0.00010224747162910903, 'epoch': 3.67}
+
 73%|███████▎  | 550/750 [59:26<20:24,  6.12s/it]
 73%|███████▎  | 551/750 [59:32<20:16,  6.11s/it]
                                                 
{'loss': 0.0994, 'grad_norm': 0.12557843327522278, 'learning_rate': 0.00010215464594349242, 'epoch': 3.67}
+
 73%|███████▎  | 551/750 [59:32<20:16,  6.11s/it]
 74%|███████▎  | 552/750 [59:38<20:09,  6.11s/it]
                                                 
{'loss': 0.1013, 'grad_norm': 0.12864509224891663, 'learning_rate': 0.00010206207261596577, 'epoch': 3.68}
+
 74%|███████▎  | 552/750 [59:38<20:09,  6.11s/it]
 74%|███████▎  | 553/750 [59:44<20:05,  6.12s/it]
                                                 
{'loss': 0.1074, 'grad_norm': 0.12279035896062851, 'learning_rate': 0.00010196975050515315, 'epoch': 3.69}
+
 74%|███████▎  | 553/750 [59:44<20:05,  6.12s/it]
 74%|███████▍  | 554/750 [59:50<20:00,  6.12s/it]
                                                 
{'loss': 0.0975, 'grad_norm': 0.11930962651968002, 'learning_rate': 0.00010187767847689286, 'epoch': 3.69}
+
 74%|███████▍  | 554/750 [59:50<20:00,  6.12s/it]
 74%|███████▍  | 555/750 [59:56<19:56,  6.14s/it]
                                                 
{'loss': 0.0866, 'grad_norm': 0.106359101831913, 'learning_rate': 0.00010178585540417863, 'epoch': 3.7}
+
 74%|███████▍  | 555/750 [59:56<19:56,  6.14s/it]
 74%|███████▍  | 556/750 [1:00:02<19:48,  6.13s/it]
                                                   
{'loss': 0.1045, 'grad_norm': 0.12246856093406677, 'learning_rate': 0.00010169428016710201, 'epoch': 3.71}
+
 74%|███████▍  | 556/750 [1:00:02<19:48,  6.13s/it]
 74%|███████▍  | 557/750 [1:00:09<19:45,  6.14s/it]
                                                   
{'loss': 0.0949, 'grad_norm': 0.11214295774698257, 'learning_rate': 0.00010160295165279474, 'epoch': 3.71}
+
 74%|███████▍  | 557/750 [1:00:09<19:45,  6.14s/it]
 74%|███████▍  | 558/750 [1:00:15<19:44,  6.17s/it]
                                                   
{'loss': 0.0884, 'grad_norm': 0.11740326881408691, 'learning_rate': 0.00010151186875537208, 'epoch': 3.72}
+
 74%|███████▍  | 558/750 [1:00:15<19:44,  6.17s/it]
 75%|███████▍  | 559/750 [1:00:21<19:32,  6.14s/it]
                                                   
{'loss': 0.0885, 'grad_norm': 0.11464753746986389, 'learning_rate': 0.00010142103037587655, 'epoch': 3.73}
+
 75%|███████▍  | 559/750 [1:00:21<19:32,  6.14s/it]
 75%|███████▍  | 560/750 [1:00:27<19:38,  6.20s/it]
                                                   
{'loss': 0.0918, 'grad_norm': 0.11156576871871948, 'learning_rate': 0.00010133043542222218, 'epoch': 3.73}
+
 75%|███████▍  | 560/750 [1:00:27<19:38,  6.20s/it]
 75%|███████▍  | 561/750 [1:00:33<19:29,  6.19s/it]
                                                   
{'loss': 0.0897, 'grad_norm': 0.11638475954532623, 'learning_rate': 0.00010124008280913954, 'epoch': 3.74}
+
 75%|███████▍  | 561/750 [1:00:33<19:29,  6.19s/it]
 75%|███████▍  | 562/750 [1:00:40<19:23,  6.19s/it]
                                                   
{'loss': 0.0863, 'grad_norm': 0.11457108706235886, 'learning_rate': 0.00010114997145812079, 'epoch': 3.75}
+
 75%|███████▍  | 562/750 [1:00:40<19:23,  6.19s/it]
 75%|███████▌  | 563/750 [1:00:46<19:10,  6.15s/it]
                                                   
{'loss': 0.0915, 'grad_norm': 0.11981626600027084, 'learning_rate': 0.000101060100297366, 'epoch': 3.75}
+
 75%|███████▌  | 563/750 [1:00:46<19:10,  6.15s/it]
 75%|███████▌  | 564/750 [1:00:52<19:02,  6.14s/it]
                                                   
{'loss': 0.0892, 'grad_norm': 0.1149091124534607, 'learning_rate': 0.00010097046826172943, 'epoch': 3.76}
+
 75%|███████▌  | 564/750 [1:00:52<19:02,  6.14s/it]
 75%|███████▌  | 565/750 [1:00:58<18:59,  6.16s/it]
                                                   
{'loss': 0.0901, 'grad_norm': 0.11524897068738937, 'learning_rate': 0.0001008810742926664, 'epoch': 3.77}
+
 75%|███████▌  | 565/750 [1:00:58<18:59,  6.16s/it]
 75%|███████▌  | 566/750 [1:01:04<18:52,  6.15s/it]
                                                   
{'loss': 0.1028, 'grad_norm': 0.11813641339540482, 'learning_rate': 0.00010079191733818102, 'epoch': 3.77}
+
 75%|███████▌  | 566/750 [1:01:04<18:52,  6.15s/it]
 76%|███████▌  | 567/750 [1:01:10<18:47,  6.16s/it]
                                                   
{'loss': 0.1016, 'grad_norm': 0.11636008322238922, 'learning_rate': 0.00010070299635277398, 'epoch': 3.78}
+
 76%|███████▌  | 567/750 [1:01:10<18:47,  6.16s/it]
 76%|███████▌  | 568/750 [1:01:17<18:44,  6.18s/it]
                                                   
{'loss': 0.1041, 'grad_norm': 0.11937959492206573, 'learning_rate': 0.00010061431029739118, 'epoch': 3.79}
+
 76%|███████▌  | 568/750 [1:01:17<18:44,  6.18s/it]
 76%|███████▌  | 569/750 [1:01:23<18:43,  6.20s/it]
                                                   
{'loss': 0.0913, 'grad_norm': 0.10870110243558884, 'learning_rate': 0.00010052585813937274, 'epoch': 3.79}
+
 76%|███████▌  | 569/750 [1:01:23<18:43,  6.20s/it]
 76%|███████▌  | 570/750 [1:01:29<18:31,  6.17s/it]
                                                   
{'loss': 0.0979, 'grad_norm': 0.12391475588083267, 'learning_rate': 0.0001004376388524024, 'epoch': 3.8}
+
 76%|███████▌  | 570/750 [1:01:29<18:31,  6.17s/it]
 76%|███████▌  | 571/750 [1:01:35<18:26,  6.18s/it]
                                                   
{'loss': 0.0932, 'grad_norm': 0.1107625961303711, 'learning_rate': 0.00010034965141645762, 'epoch': 3.81}
+
 76%|███████▌  | 571/750 [1:01:35<18:26,  6.18s/it]
 76%|███████▋  | 572/750 [1:01:41<18:17,  6.16s/it]
                                                   
{'loss': 0.1007, 'grad_norm': 0.11597322672605515, 'learning_rate': 0.00010026189481775992, 'epoch': 3.81}
+
 76%|███████▋  | 572/750 [1:01:41<18:17,  6.16s/it]
 76%|███████▋  | 573/750 [1:01:47<18:10,  6.16s/it]
                                                   
{'loss': 0.0975, 'grad_norm': 0.1213269755244255, 'learning_rate': 0.00010017436804872594, 'epoch': 3.82}
+
 76%|███████▋  | 573/750 [1:01:47<18:10,  6.16s/it]
 77%|███████▋  | 574/750 [1:01:53<17:59,  6.13s/it]
                                                   
{'loss': 0.0863, 'grad_norm': 0.11523990333080292, 'learning_rate': 0.00010008707010791883, 'epoch': 3.83}
+
 77%|███████▋  | 574/750 [1:01:53<17:59,  6.13s/it]
 77%|███████▋  | 575/750 [1:02:00<17:52,  6.13s/it]
                                                   
{'loss': 0.1002, 'grad_norm': 0.12774227559566498, 'learning_rate': 0.0001, 'epoch': 3.83}
+
 77%|███████▋  | 575/750 [1:02:00<17:52,  6.13s/it]
 77%|███████▋  | 576/750 [1:02:06<17:42,  6.11s/it]
                                                   
{'loss': 0.0903, 'grad_norm': 0.11954036355018616, 'learning_rate': 9.991315673568166e-05, 'epoch': 3.84}
+
 77%|███████▋  | 576/750 [1:02:06<17:42,  6.11s/it]
 77%|███████▋  | 577/750 [1:02:12<17:37,  6.11s/it]
                                                   
{'loss': 0.0938, 'grad_norm': 0.12281559407711029, 'learning_rate': 9.982653933167944e-05, 'epoch': 3.85}
+
 77%|███████▋  | 577/750 [1:02:12<17:37,  6.11s/it]
 77%|███████▋  | 578/750 [1:02:18<17:33,  6.13s/it]
                                                   
{'loss': 0.1012, 'grad_norm': 0.11634572595357895, 'learning_rate': 9.97401468106657e-05, 'epoch': 3.85}
+
 77%|███████▋  | 578/750 [1:02:18<17:33,  6.13s/it]
 77%|███████▋  | 579/750 [1:02:24<17:39,  6.20s/it]
                                                   
{'loss': 0.0992, 'grad_norm': 0.11939676105976105, 'learning_rate': 9.965397820122326e-05, 'epoch': 3.86}
+
 77%|███████▋  | 579/750 [1:02:24<17:39,  6.20s/it]
 77%|███████▋  | 580/750 [1:02:31<17:36,  6.21s/it]
                                                   
{'loss': 0.08, 'grad_norm': 0.10871097445487976, 'learning_rate': 9.956803253779939e-05, 'epoch': 3.87}
+
 77%|███████▋  | 580/750 [1:02:31<17:36,  6.21s/it]
 77%|███████▋  | 581/750 [1:02:37<17:25,  6.19s/it]
                                                   
{'loss': 0.0946, 'grad_norm': 0.11240905523300171, 'learning_rate': 9.94823088606605e-05, 'epoch': 3.87}
+
 77%|███████▋  | 581/750 [1:02:37<17:25,  6.19s/it]
 78%|███████▊  | 582/750 [1:02:43<17:20,  6.19s/it]
                                                   
{'loss': 0.1009, 'grad_norm': 0.10966379195451736, 'learning_rate': 9.939680621584706e-05, 'epoch': 3.88}
+
 78%|███████▊  | 582/750 [1:02:43<17:20,  6.19s/it]
 78%|███████▊  | 583/750 [1:02:49<17:12,  6.18s/it]
                                                   
{'loss': 0.1085, 'grad_norm': 0.1261599361896515, 'learning_rate': 9.931152365512906e-05, 'epoch': 3.89}
+
 78%|███████▊  | 583/750 [1:02:49<17:12,  6.18s/it]
 78%|███████▊  | 584/750 [1:02:55<17:03,  6.17s/it]
                                                   
{'loss': 0.0964, 'grad_norm': 0.11226785182952881, 'learning_rate': 9.922646023596178e-05, 'epoch': 3.89}
+
 78%|███████▊  | 584/750 [1:02:55<17:03,  6.17s/it]
 78%|███████▊  | 585/750 [1:03:01<16:53,  6.14s/it]
                                                   
{'loss': 0.0832, 'grad_norm': 0.11634642630815506, 'learning_rate': 9.914161502144209e-05, 'epoch': 3.9}
+
 78%|███████▊  | 585/750 [1:03:01<16:53,  6.14s/it]
 78%|███████▊  | 586/750 [1:03:07<16:44,  6.13s/it]
                                                   
{'loss': 0.0999, 'grad_norm': 0.13422991335391998, 'learning_rate': 9.905698708026503e-05, 'epoch': 3.91}
+
 78%|███████▊  | 586/750 [1:03:07<16:44,  6.13s/it]
 78%|███████▊  | 587/750 [1:03:14<16:43,  6.16s/it]
                                                   
{'loss': 0.0991, 'grad_norm': 0.11260322481393814, 'learning_rate': 9.897257548668106e-05, 'epoch': 3.91}
+
 78%|███████▊  | 587/750 [1:03:14<16:43,  6.16s/it]
 78%|███████▊  | 588/750 [1:03:20<16:34,  6.14s/it]
                                                   
{'loss': 0.0882, 'grad_norm': 0.11597412824630737, 'learning_rate': 9.888837932045328e-05, 'epoch': 3.92}
+
 78%|███████▊  | 588/750 [1:03:20<16:34,  6.14s/it]
 79%|███████▊  | 589/750 [1:03:26<16:27,  6.14s/it]
                                                   
{'loss': 0.1225, 'grad_norm': 0.1273944079875946, 'learning_rate': 9.880439766681545e-05, 'epoch': 3.93}
+
 79%|███████▊  | 589/750 [1:03:26<16:27,  6.14s/it]
 79%|███████▊  | 590/750 [1:03:32<16:22,  6.14s/it]
                                                   
{'loss': 0.0933, 'grad_norm': 0.11119824647903442, 'learning_rate': 9.872062961643026e-05, 'epoch': 3.93}
+
 79%|███████▊  | 590/750 [1:03:32<16:22,  6.14s/it]
 79%|███████▉  | 591/750 [1:03:38<16:20,  6.17s/it]
                                                   
{'loss': 0.1004, 'grad_norm': 0.1181047111749649, 'learning_rate': 9.863707426534786e-05, 'epoch': 3.94}
+
 79%|███████▉  | 591/750 [1:03:38<16:20,  6.17s/it]
 79%|███████▉  | 592/750 [1:03:44<16:11,  6.15s/it]
                                                   
{'loss': 0.1046, 'grad_norm': 0.12149988114833832, 'learning_rate': 9.8553730714965e-05, 'epoch': 3.95}
+
 79%|███████▉  | 592/750 [1:03:44<16:11,  6.15s/it]
 79%|███████▉  | 593/750 [1:03:50<16:04,  6.14s/it]
                                                   
{'loss': 0.1082, 'grad_norm': 0.1222451850771904, 'learning_rate': 9.847059807198442e-05, 'epoch': 3.95}
+
 79%|███████▉  | 593/750 [1:03:50<16:04,  6.14s/it]
 79%|███████▉  | 594/750 [1:03:57<16:02,  6.17s/it]
                                                   
{'loss': 0.1075, 'grad_norm': 0.12219641357660294, 'learning_rate': 9.838767544837452e-05, 'epoch': 3.96}
+
 79%|███████▉  | 594/750 [1:03:57<16:02,  6.17s/it]
 79%|███████▉  | 595/750 [1:04:03<15:52,  6.14s/it]
                                                   
{'loss': 0.0952, 'grad_norm': 0.1670890897512436, 'learning_rate': 9.830496196132975e-05, 'epoch': 3.97}
+
 79%|███████▉  | 595/750 [1:04:03<15:52,  6.14s/it]
 79%|███████▉  | 596/750 [1:04:09<15:46,  6.15s/it]
                                                   
{'loss': 0.0981, 'grad_norm': 0.11860240250825882, 'learning_rate': 9.822245673323086e-05, 'epoch': 3.97}
+
 79%|███████▉  | 596/750 [1:04:09<15:46,  6.15s/it]
 80%|███████▉  | 597/750 [1:04:15<15:50,  6.21s/it]
                                                   
{'loss': 0.0955, 'grad_norm': 0.11549799144268036, 'learning_rate': 9.814015889160605e-05, 'epoch': 3.98}
+
 80%|███████▉  | 597/750 [1:04:15<15:50,  6.21s/it]
 80%|███████▉  | 598/750 [1:04:21<15:36,  6.16s/it]
                                                   
{'loss': 0.0953, 'grad_norm': 0.11920598149299622, 'learning_rate': 9.805806756909202e-05, 'epoch': 3.99}
+
 80%|███████▉  | 598/750 [1:04:21<15:36,  6.16s/it]
 80%|███████▉  | 599/750 [1:04:27<15:31,  6.17s/it]
                                                   
{'loss': 0.1029, 'grad_norm': 0.12221633642911911, 'learning_rate': 9.797618190339569e-05, 'epoch': 3.99}
+
 80%|███████▉  | 599/750 [1:04:27<15:31,  6.17s/it]
 80%|████████  | 600/750 [1:04:36<16:50,  6.74s/it]
                                                   
{'loss': 0.0852, 'grad_norm': 0.11316592991352081, 'learning_rate': 9.789450103725609e-05, 'epoch': 4.0}
+
 80%|████████  | 600/750 [1:04:36<16:50,  6.74s/it][INFO|trainer.py:3831] 2025-06-26 23:20:29,213 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-26 23:20:29,213 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-26 23:20:29,213 >>   Batch size = 25
+
+
  0%|          | 0/10 [00:00<?, ?it/s]
+
 20%|██        | 2/10 [00:01<00:07,  1.09it/s]
+
 30%|███       | 3/10 [00:03<00:09,  1.31s/it]
+
 40%|████      | 4/10 [00:05<00:09,  1.51s/it]
+
 50%|█████     | 5/10 [00:07<00:08,  1.63s/it]
+
 60%|██████    | 6/10 [00:09<00:06,  1.70s/it]
+
 70%|███████   | 7/10 [00:11<00:05,  1.74s/it]
+
 80%|████████  | 8/10 [00:12<00:03,  1.77s/it]
+
 90%|█████████ | 9/10 [00:14<00:01,  1.80s/it]
+
100%|██████████| 10/10 [00:18<00:00,  2.39s/it]
                                                   
+
                                               
{'eval_loss': 0.14695635437965393, 'eval_accuracy': 0.2945686370297997, 'eval_runtime': 34.4502, 'eval_samples_per_second': 29.027, 'eval_steps_per_second': 0.29, 'epoch': 4.0}
+
 80%|████████  | 600/750 [1:05:10<16:50,  6.74s/it]
+
100%|██████████| 10/10 [00:24<00:00,  2.39s/it]
+
                                               /home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[INFO|trainer.py:3515] 2025-06-26 23:21:17,283 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-600
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8895-211b866e1856ee162b9b984e;76a667cd-c953-4f09-8446-a6b8577d3fc4)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-26 23:21:17,668 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-600/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-26 23:21:17,668 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-600/special_tokens_map.json
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user. 
+  warnings.warn(  # warn only once
+[2025-06-26 23:21:18,886] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Checkpoint global_step600 is begin to save!
+[2025-06-26 23:21:18,910] [INFO] [logging.py:107:log_dist] [Rank 0] Saving model checkpoint: ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-600/global_step600/mp_rank_00_model_states.pt
+[INFO|trainer.py:3607] 2025-06-26 23:21:19,037 >> Deleting older checkpoint [outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-200] due to args.save_total_limit
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8897-46eb4180178ab85d69a88c0e;d2e4ffb3-e516-40f3-9899-ecd04972acc8)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8897-5c192e2d260032f91caa6e7c;0a0fb35e-2e26-40b2-ac2f-4888a404b445)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8897-5f003e30467257235e444f17;64b508d6-f1bc-438b-8ac9-752548bb8b86)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8897-5681373c21c7fe91464ac624;1b5ff3df-ffb4-4af3-ba8b-d7142c09762a)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/save_and_load.py:236: UserWarning: Could not find a config file in meta-llama/Llama-3.1-8B-Instruct - will assume that the vocabulary was not modified.
+  warnings.warn(
+
 80%|████████  | 601/750 [1:05:40<59:52, 24.11s/it]
                                                   
{'loss': 0.0859, 'grad_norm': 0.11321733891963959, 'learning_rate': 9.781302411840674e-05, 'epoch': 4.01}
+
 80%|████████  | 601/750 [1:05:40<59:52, 24.11s/it]
 80%|████████  | 602/750 [1:05:46<46:07, 18.70s/it]
                                                   
{'loss': 0.082, 'grad_norm': 0.11039352416992188, 'learning_rate': 9.773175029953825e-05, 'epoch': 4.01}
+
 80%|████████  | 602/750 [1:05:46<46:07, 18.70s/it]
 80%|████████  | 603/750 [1:05:52<36:34, 14.93s/it]
                                                   
{'loss': 0.0696, 'grad_norm': 0.11571627110242844, 'learning_rate': 9.76506787382613e-05, 'epoch': 4.02}
+
 80%|████████  | 603/750 [1:05:52<36:34, 14.93s/it]
 81%|████████  | 604/750 [1:05:59<29:55, 12.30s/it]
                                                   
{'loss': 0.0754, 'grad_norm': 0.11830026656389236, 'learning_rate': 9.756980859707e-05, 'epoch': 4.03}
+
 81%|████████  | 604/750 [1:05:59<29:55, 12.30s/it]
 81%|████████  | 605/750 [1:06:05<25:14, 10.45s/it]
                                                   
{'loss': 0.0775, 'grad_norm': 0.13570784032344818, 'learning_rate': 9.748913904330553e-05, 'epoch': 4.03}
+
 81%|████████  | 605/750 [1:06:05<25:14, 10.45s/it]
 81%|████████  | 606/750 [1:06:11<22:00,  9.17s/it]
                                                   
{'loss': 0.0781, 'grad_norm': 0.14153192937374115, 'learning_rate': 9.740866924912017e-05, 'epoch': 4.04}
+
 81%|████████  | 606/750 [1:06:11<22:00,  9.17s/it]
 81%|████████  | 607/750 [1:06:17<19:44,  8.28s/it]
                                                   
{'loss': 0.078, 'grad_norm': 0.1518002450466156, 'learning_rate': 9.732839839144154e-05, 'epoch': 4.05}
+
 81%|████████  | 607/750 [1:06:17<19:44,  8.28s/it]
 81%|████████  | 608/750 [1:06:23<18:03,  7.63s/it]
                                                   
{'loss': 0.0729, 'grad_norm': 0.1601392924785614, 'learning_rate': 9.724832565193738e-05, 'epoch': 4.05}
+
 81%|████████  | 608/750 [1:06:23<18:03,  7.63s/it]
 81%|████████  | 609/750 [1:06:29<16:53,  7.19s/it]
                                                   
{'loss': 0.0778, 'grad_norm': 0.14252175390720367, 'learning_rate': 9.716845021698033e-05, 'epoch': 4.06}
+
 81%|████████  | 609/750 [1:06:29<16:53,  7.19s/it]
 81%|████████▏ | 610/750 [1:06:35<16:02,  6.88s/it]
                                                   
{'loss': 0.0692, 'grad_norm': 0.1270168572664261, 'learning_rate': 9.708877127761337e-05, 'epoch': 4.07}
+
 81%|████████▏ | 610/750 [1:06:35<16:02,  6.88s/it]
 81%|████████▏ | 611/750 [1:06:42<15:27,  6.67s/it]
                                                   
{'loss': 0.0701, 'grad_norm': 0.11871245503425598, 'learning_rate': 9.700928802951527e-05, 'epoch': 4.07}
+
 81%|████████▏ | 611/750 [1:06:42<15:27,  6.67s/it]
 82%|████████▏ | 612/750 [1:06:48<14:59,  6.52s/it]
                                                   
{'loss': 0.0792, 'grad_norm': 0.12868107855319977, 'learning_rate': 9.69299996729666e-05, 'epoch': 4.08}
+
 82%|████████▏ | 612/750 [1:06:48<14:59,  6.52s/it]
 82%|████████▏ | 613/750 [1:06:54<14:39,  6.42s/it]
                                                   
{'loss': 0.0763, 'grad_norm': 0.12250402569770813, 'learning_rate': 9.685090541281594e-05, 'epoch': 4.09}
+
 82%|████████▏ | 613/750 [1:06:54<14:39,  6.42s/it]
 82%|████████▏ | 614/750 [1:07:00<14:22,  6.34s/it]
                                                   
{'loss': 0.0709, 'grad_norm': 0.12387514859437943, 'learning_rate': 9.677200445844624e-05, 'epoch': 4.09}
+
 82%|████████▏ | 614/750 [1:07:00<14:22,  6.34s/it]
 82%|████████▏ | 615/750 [1:07:06<14:10,  6.30s/it]
                                                   
{'loss': 0.0739, 'grad_norm': 0.12223586440086365, 'learning_rate': 9.669329602374179e-05, 'epoch': 4.1}
+
 82%|████████▏ | 615/750 [1:07:06<14:10,  6.30s/it]
 82%|████████▏ | 616/750 [1:07:13<14:10,  6.35s/it]
                                                   
{'loss': 0.0745, 'grad_norm': 0.12986072897911072, 'learning_rate': 9.661477932705526e-05, 'epoch': 4.11}
+
 82%|████████▏ | 616/750 [1:07:13<14:10,  6.35s/it]
 82%|████████▏ | 617/750 [1:07:19<13:58,  6.30s/it]
                                                   
{'loss': 0.0853, 'grad_norm': 0.13613200187683105, 'learning_rate': 9.65364535911751e-05, 'epoch': 4.11}
+
 82%|████████▏ | 617/750 [1:07:19<13:58,  6.30s/it]
 82%|████████▏ | 618/750 [1:07:25<13:54,  6.32s/it]
                                                   
{'loss': 0.0676, 'grad_norm': 0.17618970572948456, 'learning_rate': 9.645831804329319e-05, 'epoch': 4.12}
+
 82%|████████▏ | 618/750 [1:07:25<13:54,  6.32s/it]
 83%|████████▎ | 619/750 [1:07:32<13:39,  6.26s/it]
                                                   
{'loss': 0.0793, 'grad_norm': 0.1486322432756424, 'learning_rate': 9.63803719149729e-05, 'epoch': 4.13}
+
 83%|████████▎ | 619/750 [1:07:32<13:39,  6.26s/it]
 83%|████████▎ | 620/750 [1:07:38<13:32,  6.25s/it]
                                                   
{'loss': 0.0724, 'grad_norm': 0.14255782961845398, 'learning_rate': 9.630261444211729e-05, 'epoch': 4.13}
+
 83%|████████▎ | 620/750 [1:07:38<13:32,  6.25s/it]
 83%|████████▎ | 621/750 [1:07:44<13:22,  6.22s/it]
                                                   
{'loss': 0.081, 'grad_norm': 0.14457444846630096, 'learning_rate': 9.622504486493763e-05, 'epoch': 4.14}
+
 83%|████████▎ | 621/750 [1:07:44<13:22,  6.22s/it]
 83%|████████▎ | 622/750 [1:07:50<13:17,  6.23s/it]
                                                   
{'loss': 0.0697, 'grad_norm': 0.13083262741565704, 'learning_rate': 9.614766242792228e-05, 'epoch': 4.15}
+
 83%|████████▎ | 622/750 [1:07:50<13:17,  6.23s/it]
 83%|████████▎ | 623/750 [1:07:56<13:10,  6.22s/it]
                                                   
{'loss': 0.0777, 'grad_norm': 0.13435541093349457, 'learning_rate': 9.607046637980575e-05, 'epoch': 4.15}
+
 83%|████████▎ | 623/750 [1:07:56<13:10,  6.22s/it]
 83%|████████▎ | 624/750 [1:08:03<13:02,  6.21s/it]
                                                   
{'loss': 0.0723, 'grad_norm': 0.12967945635318756, 'learning_rate': 9.599345597353806e-05, 'epoch': 4.16}
+
 83%|████████▎ | 624/750 [1:08:03<13:02,  6.21s/it]
 83%|████████▎ | 625/750 [1:08:09<12:55,  6.20s/it]
                                                   
{'loss': 0.0722, 'grad_norm': 0.12792378664016724, 'learning_rate': 9.59166304662544e-05, 'epoch': 4.17}
+
 83%|████████▎ | 625/750 [1:08:09<12:55,  6.20s/it]
 83%|████████▎ | 626/750 [1:08:15<12:47,  6.19s/it]
                                                   
{'loss': 0.0847, 'grad_norm': 0.1359768658876419, 'learning_rate': 9.5839989119245e-05, 'epoch': 4.17}
+
 83%|████████▎ | 626/750 [1:08:15<12:47,  6.19s/it]
 84%|████████▎ | 627/750 [1:08:21<12:41,  6.19s/it]
                                                   
{'loss': 0.0743, 'grad_norm': 0.13040956854820251, 'learning_rate': 9.57635311979253e-05, 'epoch': 4.18}
+
 84%|████████▎ | 627/750 [1:08:21<12:41,  6.19s/it]
 84%|████████▎ | 628/750 [1:08:27<12:32,  6.17s/it]
                                                   
{'loss': 0.0765, 'grad_norm': 0.13199128210544586, 'learning_rate': 9.568725597180648e-05, 'epoch': 4.19}
+
 84%|████████▎ | 628/750 [1:08:27<12:32,  6.17s/it]
 84%|████████▍ | 629/750 [1:08:33<12:28,  6.19s/it]
                                                   
{'loss': 0.0654, 'grad_norm': 0.1243375688791275, 'learning_rate': 9.561116271446594e-05, 'epoch': 4.19}
+
 84%|████████▍ | 629/750 [1:08:33<12:28,  6.19s/it]
 84%|████████▍ | 630/750 [1:08:40<12:23,  6.19s/it]
                                                   
{'loss': 0.0639, 'grad_norm': 0.12788942456245422, 'learning_rate': 9.553525070351848e-05, 'epoch': 4.2}
+
 84%|████████▍ | 630/750 [1:08:40<12:23,  6.19s/it]
 84%|████████▍ | 631/750 [1:08:46<12:17,  6.20s/it]
                                                   
{'loss': 0.0621, 'grad_norm': 0.13074801862239838, 'learning_rate': 9.54595192205873e-05, 'epoch': 4.21}
+
 84%|████████▍ | 631/750 [1:08:46<12:17,  6.20s/it]
 84%|████████▍ | 632/750 [1:08:52<12:12,  6.21s/it]
                                                   
{'loss': 0.0822, 'grad_norm': 0.15035344660282135, 'learning_rate': 9.538396755127553e-05, 'epoch': 4.21}
+
 84%|████████▍ | 632/750 [1:08:52<12:12,  6.21s/it]
 84%|████████▍ | 633/750 [1:08:58<12:03,  6.19s/it]
                                                   
{'loss': 0.0936, 'grad_norm': 0.17540502548217773, 'learning_rate': 9.530859498513802e-05, 'epoch': 4.22}
+
 84%|████████▍ | 633/750 [1:08:58<12:03,  6.19s/it]
 85%|████████▍ | 634/750 [1:09:05<12:02,  6.23s/it]
                                                   
{'loss': 0.0714, 'grad_norm': 0.15226072072982788, 'learning_rate': 9.523340081565305e-05, 'epoch': 4.23}
+
 85%|████████▍ | 634/750 [1:09:05<12:02,  6.23s/it]
 85%|████████▍ | 635/750 [1:09:11<11:55,  6.22s/it]
                                                   
{'loss': 0.0838, 'grad_norm': 0.14751319587230682, 'learning_rate': 9.515838434019473e-05, 'epoch': 4.23}
+
 85%|████████▍ | 635/750 [1:09:11<11:55,  6.22s/it]
 85%|████████▍ | 636/750 [1:09:17<11:49,  6.23s/it]
                                                   
{'loss': 0.0726, 'grad_norm': 0.13842786848545074, 'learning_rate': 9.508354486000538e-05, 'epoch': 4.24}
+
 85%|████████▍ | 636/750 [1:09:17<11:49,  6.23s/it]
 85%|████████▍ | 637/750 [1:09:23<11:46,  6.25s/it]
                                                   
{'loss': 0.0695, 'grad_norm': 0.12967897951602936, 'learning_rate': 9.500888168016802e-05, 'epoch': 4.25}
+
 85%|████████▍ | 637/750 [1:09:23<11:46,  6.25s/it]
 85%|████████▌ | 638/750 [1:09:29<11:35,  6.21s/it]
                                                   
{'loss': 0.0779, 'grad_norm': 0.1405758261680603, 'learning_rate': 9.493439410957959e-05, 'epoch': 4.25}
+
 85%|████████▌ | 638/750 [1:09:29<11:35,  6.21s/it]
 85%|████████▌ | 639/750 [1:09:36<11:28,  6.20s/it]
                                                   
{'loss': 0.0824, 'grad_norm': 0.14884178340435028, 'learning_rate': 9.48600814609237e-05, 'epoch': 4.26}
+
 85%|████████▌ | 639/750 [1:09:36<11:28,  6.20s/it]
 85%|████████▌ | 640/750 [1:09:42<11:21,  6.19s/it]
                                                   
{'loss': 0.0806, 'grad_norm': 0.14168286323547363, 'learning_rate': 9.478594305064439e-05, 'epoch': 4.27}
+
 85%|████████▌ | 640/750 [1:09:42<11:21,  6.19s/it]
 85%|████████▌ | 641/750 [1:09:48<11:13,  6.18s/it]
                                                   
{'loss': 0.0788, 'grad_norm': 0.1437740921974182, 'learning_rate': 9.471197819891936e-05, 'epoch': 4.27}
+
 85%|████████▌ | 641/750 [1:09:48<11:13,  6.18s/it]
 86%|████████▌ | 642/750 [1:09:54<11:06,  6.17s/it]
                                                   
{'loss': 0.0783, 'grad_norm': 0.14915212988853455, 'learning_rate': 9.463818622963402e-05, 'epoch': 4.28}
+
 86%|████████▌ | 642/750 [1:09:54<11:06,  6.17s/it]
 86%|████████▌ | 643/750 [1:10:00<11:00,  6.17s/it]
                                                   
{'loss': 0.0769, 'grad_norm': 0.15485066175460815, 'learning_rate': 9.45645664703555e-05, 'epoch': 4.29}
+
 86%|████████▌ | 643/750 [1:10:00<11:00,  6.17s/it]
 86%|████████▌ | 644/750 [1:10:06<10:53,  6.17s/it]
                                                   
{'loss': 0.0792, 'grad_norm': 0.15206070244312286, 'learning_rate': 9.44911182523068e-05, 'epoch': 4.29}
+
 86%|████████▌ | 644/750 [1:10:06<10:53,  6.17s/it]
 86%|████████▌ | 645/750 [1:10:13<10:46,  6.15s/it]
                                                   
{'loss': 0.0687, 'grad_norm': 0.14829127490520477, 'learning_rate': 9.441784091034141e-05, 'epoch': 4.3}
+
 86%|████████▌ | 645/750 [1:10:13<10:46,  6.15s/it]
 86%|████████▌ | 646/750 [1:10:19<10:42,  6.18s/it]
                                                   
{'loss': 0.0748, 'grad_norm': 0.13177165389060974, 'learning_rate': 9.434473378291795e-05, 'epoch': 4.31}
+
 86%|████████▌ | 646/750 [1:10:19<10:42,  6.18s/it]
 86%|████████▋ | 647/750 [1:10:25<10:34,  6.16s/it]
                                                   
{'loss': 0.0761, 'grad_norm': 0.1434299349784851, 'learning_rate': 9.427179621207505e-05, 'epoch': 4.31}
+
 86%|████████▋ | 647/750 [1:10:25<10:34,  6.16s/it]
 86%|████████▋ | 648/750 [1:10:31<10:27,  6.16s/it]
                                                   
{'loss': 0.0835, 'grad_norm': 0.1464683711528778, 'learning_rate': 9.419902754340649e-05, 'epoch': 4.32}
+
 86%|████████▋ | 648/750 [1:10:31<10:27,  6.16s/it]
 87%|████████▋ | 649/750 [1:10:37<10:22,  6.16s/it]
                                                   
{'loss': 0.0836, 'grad_norm': 0.14544245600700378, 'learning_rate': 9.41264271260367e-05, 'epoch': 4.33}
+
 87%|████████▋ | 649/750 [1:10:37<10:22,  6.16s/it]
 87%|████████▋ | 650/750 [1:10:43<10:16,  6.16s/it]
                                                   
{'loss': 0.0664, 'grad_norm': 0.1314661204814911, 'learning_rate': 9.405399431259603e-05, 'epoch': 4.33}
+
 87%|████████▋ | 650/750 [1:10:43<10:16,  6.16s/it]
 87%|████████▋ | 651/750 [1:10:49<10:08,  6.15s/it]
                                                   
{'loss': 0.0809, 'grad_norm': 0.13976162672042847, 'learning_rate': 9.398172845919671e-05, 'epoch': 4.34}
+
 87%|████████▋ | 651/750 [1:10:49<10:08,  6.15s/it]
 87%|████████▋ | 652/750 [1:10:56<10:00,  6.13s/it]
                                                   
{'loss': 0.083, 'grad_norm': 0.14182595908641815, 'learning_rate': 9.390962892540881e-05, 'epoch': 4.35}
+
 87%|████████▋ | 652/750 [1:10:56<10:00,  6.13s/it]
 87%|████████▋ | 653/750 [1:11:02<09:58,  6.17s/it]
                                                   
{'loss': 0.0798, 'grad_norm': 0.1485460102558136, 'learning_rate': 9.383769507423618e-05, 'epoch': 4.35}
+
 87%|████████▋ | 653/750 [1:11:02<09:58,  6.17s/it]
 87%|████████▋ | 654/750 [1:11:08<09:51,  6.16s/it]
                                                   
{'loss': 0.0827, 'grad_norm': 0.1494467407464981, 'learning_rate': 9.376592627209315e-05, 'epoch': 4.36}
+
 87%|████████▋ | 654/750 [1:11:08<09:51,  6.16s/it]
 87%|████████▋ | 655/750 [1:11:14<09:47,  6.19s/it]
                                                   
{'loss': 0.072, 'grad_norm': 0.13865183293819427, 'learning_rate': 9.369432188878082e-05, 'epoch': 4.37}
+
 87%|████████▋ | 655/750 [1:11:14<09:47,  6.19s/it]
 87%|████████▋ | 656/750 [1:11:20<09:39,  6.16s/it]
                                                   
{'loss': 0.0754, 'grad_norm': 0.14116638898849487, 'learning_rate': 9.362288129746395e-05, 'epoch': 4.37}
+
 87%|████████▋ | 656/750 [1:11:20<09:39,  6.16s/it]
 88%|████████▊ | 657/750 [1:11:26<09:31,  6.15s/it]
                                                   
{'loss': 0.0781, 'grad_norm': 0.1584867686033249, 'learning_rate': 9.355160387464784e-05, 'epoch': 4.38}
+
 88%|████████▊ | 657/750 [1:11:26<09:31,  6.15s/it]
 88%|████████▊ | 658/750 [1:11:33<09:25,  6.15s/it]
                                                   
{'loss': 0.0759, 'grad_norm': 0.14271670579910278, 'learning_rate': 9.348048900015556e-05, 'epoch': 4.39}
+
 88%|████████▊ | 658/750 [1:11:33<09:25,  6.15s/it]
 88%|████████▊ | 659/750 [1:11:39<09:20,  6.16s/it]
                                                   
{'loss': 0.0768, 'grad_norm': 0.1325335055589676, 'learning_rate': 9.340953605710524e-05, 'epoch': 4.39}
+
 88%|████████▊ | 659/750 [1:11:39<09:20,  6.16s/it]
 88%|████████▊ | 660/750 [1:11:45<09:13,  6.15s/it]
                                                   
{'loss': 0.0816, 'grad_norm': 0.146781325340271, 'learning_rate': 9.333874443188752e-05, 'epoch': 4.4}
+
 88%|████████▊ | 660/750 [1:11:45<09:13,  6.15s/it]
 88%|████████▊ | 661/750 [1:11:51<09:09,  6.17s/it]
                                                   
{'loss': 0.0816, 'grad_norm': 0.14144185185432434, 'learning_rate': 9.326811351414343e-05, 'epoch': 4.41}
+
 88%|████████▊ | 661/750 [1:11:51<09:09,  6.17s/it]
 88%|████████▊ | 662/750 [1:11:57<09:05,  6.20s/it]
                                                   
{'loss': 0.0754, 'grad_norm': 0.14317448437213898, 'learning_rate': 9.319764269674211e-05, 'epoch': 4.41}
+
 88%|████████▊ | 662/750 [1:11:57<09:05,  6.20s/it]
 88%|████████▊ | 663/750 [1:12:04<08:57,  6.18s/it]
                                                   
{'loss': 0.0824, 'grad_norm': 0.14584535360336304, 'learning_rate': 9.312733137575907e-05, 'epoch': 4.42}
+
 88%|████████▊ | 663/750 [1:12:04<08:57,  6.18s/it]
 89%|████████▊ | 664/750 [1:12:10<08:50,  6.17s/it]
                                                   
{'loss': 0.0824, 'grad_norm': 0.1390840709209442, 'learning_rate': 9.305717895045427e-05, 'epoch': 4.43}
+
 89%|████████▊ | 664/750 [1:12:10<08:50,  6.17s/it]
 89%|████████▊ | 665/750 [1:12:16<08:43,  6.16s/it]
                                                   
{'loss': 0.0772, 'grad_norm': 0.1434590071439743, 'learning_rate': 9.298718482325068e-05, 'epoch': 4.43}
+
 89%|████████▊ | 665/750 [1:12:16<08:43,  6.16s/it]
 89%|████████▉ | 666/750 [1:12:22<08:36,  6.15s/it]
                                                   
{'loss': 0.0887, 'grad_norm': 0.16114962100982666, 'learning_rate': 9.291734839971293e-05, 'epoch': 4.44}
+
 89%|████████▉ | 666/750 [1:12:22<08:36,  6.15s/it]
 89%|████████▉ | 667/750 [1:12:28<08:30,  6.15s/it]
                                                   
{'loss': 0.0853, 'grad_norm': 0.13922066986560822, 'learning_rate': 9.284766908852593e-05, 'epoch': 4.45}
+
 89%|████████▉ | 667/750 [1:12:28<08:30,  6.15s/it]
 89%|████████▉ | 668/750 [1:12:34<08:24,  6.15s/it]
                                                   
{'loss': 0.0685, 'grad_norm': 0.13378609716892242, 'learning_rate': 9.277814630147407e-05, 'epoch': 4.45}
+
 89%|████████▉ | 668/750 [1:12:34<08:24,  6.15s/it]
 89%|████████▉ | 669/750 [1:12:40<08:17,  6.14s/it]
                                                   
{'loss': 0.0796, 'grad_norm': 0.13879472017288208, 'learning_rate': 9.270877945342015e-05, 'epoch': 4.46}
+
 89%|████████▉ | 669/750 [1:12:40<08:17,  6.14s/it]
 89%|████████▉ | 670/750 [1:12:47<08:11,  6.14s/it]
                                                   
{'loss': 0.071, 'grad_norm': 0.1322331577539444, 'learning_rate': 9.263956796228493e-05, 'epoch': 4.47}
+
 89%|████████▉ | 670/750 [1:12:47<08:11,  6.14s/it]
 89%|████████▉ | 671/750 [1:12:53<08:09,  6.19s/it]
                                                   
{'loss': 0.0766, 'grad_norm': 0.14124593138694763, 'learning_rate': 9.257051124902634e-05, 'epoch': 4.47}
+
 89%|████████▉ | 671/750 [1:12:53<08:09,  6.19s/it]
 90%|████████▉ | 672/750 [1:12:59<08:01,  6.18s/it]
                                                   
{'loss': 0.0802, 'grad_norm': 0.14281946420669556, 'learning_rate': 9.250160873761937e-05, 'epoch': 4.48}
+
 90%|████████▉ | 672/750 [1:12:59<08:01,  6.18s/it]
 90%|████████▉ | 673/750 [1:13:05<07:54,  6.17s/it]
                                                   
{'loss': 0.0746, 'grad_norm': 0.13686195015907288, 'learning_rate': 9.243285985503578e-05, 'epoch': 4.49}
+
 90%|████████▉ | 673/750 [1:13:05<07:54,  6.17s/it]
 90%|████████▉ | 674/750 [1:13:11<07:50,  6.19s/it]
                                                   
{'loss': 0.0733, 'grad_norm': 0.1372523456811905, 'learning_rate': 9.236426403122409e-05, 'epoch': 4.49}
+
 90%|████████▉ | 674/750 [1:13:11<07:50,  6.19s/it]
 90%|█████████ | 675/750 [1:13:18<07:44,  6.19s/it]
                                                   
{'loss': 0.0691, 'grad_norm': 0.21261753141880035, 'learning_rate': 9.229582069908974e-05, 'epoch': 4.5}
+
 90%|█████████ | 675/750 [1:13:18<07:44,  6.19s/it]
 90%|█████████ | 676/750 [1:13:24<07:37,  6.18s/it]
                                                   
{'loss': 0.084, 'grad_norm': 0.1425894796848297, 'learning_rate': 9.222752929447538e-05, 'epoch': 4.51}
+
 90%|█████████ | 676/750 [1:13:24<07:37,  6.18s/it]
 90%|█████████ | 677/750 [1:13:30<07:31,  6.19s/it]
                                                   
{'loss': 0.074, 'grad_norm': 0.13595019280910492, 'learning_rate': 9.215938925614141e-05, 'epoch': 4.51}
+
 90%|█████████ | 677/750 [1:13:30<07:31,  6.19s/it]
 90%|█████████ | 678/750 [1:13:36<07:24,  6.18s/it]
                                                   
{'loss': 0.0698, 'grad_norm': 0.1334165781736374, 'learning_rate': 9.209140002574652e-05, 'epoch': 4.52}
+
 90%|█████████ | 678/750 [1:13:36<07:24,  6.18s/it]
 91%|█████████ | 679/750 [1:13:42<07:17,  6.17s/it]
                                                   
{'loss': 0.0822, 'grad_norm': 0.14911916851997375, 'learning_rate': 9.20235610478285e-05, 'epoch': 4.53}
+
 91%|█████████ | 679/750 [1:13:42<07:17,  6.17s/it]
 91%|█████████ | 680/750 [1:13:48<07:11,  6.16s/it]
                                                   
{'loss': 0.0771, 'grad_norm': 0.1393679827451706, 'learning_rate': 9.19558717697852e-05, 'epoch': 4.53}
+
 91%|█████████ | 680/750 [1:13:48<07:11,  6.16s/it]
 91%|█████████ | 681/750 [1:13:54<07:03,  6.13s/it]
                                                   
{'loss': 0.072, 'grad_norm': 0.13874053955078125, 'learning_rate': 9.18883316418556e-05, 'epoch': 4.54}
+
 91%|█████████ | 681/750 [1:13:54<07:03,  6.13s/it]
 91%|█████████ | 682/750 [1:14:01<06:58,  6.15s/it]
                                                   
{'loss': 0.0715, 'grad_norm': 0.1403961032629013, 'learning_rate': 9.182094011710111e-05, 'epoch': 4.55}
+
 91%|█████████ | 682/750 [1:14:01<06:58,  6.15s/it]
 91%|█████████ | 683/750 [1:14:07<06:53,  6.17s/it]
                                                   
{'loss': 0.0796, 'grad_norm': 0.1321626454591751, 'learning_rate': 9.175369665138691e-05, 'epoch': 4.55}
+
 91%|█████████ | 683/750 [1:14:07<06:53,  6.17s/it]
 91%|█████████ | 684/750 [1:14:13<06:47,  6.17s/it]
                                                   
{'loss': 0.0735, 'grad_norm': 0.14288955926895142, 'learning_rate': 9.168660070336348e-05, 'epoch': 4.56}
+
 91%|█████████ | 684/750 [1:14:13<06:47,  6.17s/it]
 91%|█████████▏| 685/750 [1:14:19<06:39,  6.14s/it]
                                                   
{'loss': 0.0789, 'grad_norm': 0.1473475992679596, 'learning_rate': 9.161965173444834e-05, 'epoch': 4.57}
+
 91%|█████████▏| 685/750 [1:14:19<06:39,  6.14s/it]
 91%|█████████▏| 686/750 [1:14:25<06:32,  6.13s/it]
                                                   
{'loss': 0.0939, 'grad_norm': 0.16288213431835175, 'learning_rate': 9.155284920880796e-05, 'epoch': 4.57}
+
 91%|█████████▏| 686/750 [1:14:25<06:32,  6.13s/it]
 92%|█████████▏| 687/750 [1:14:31<06:27,  6.15s/it]
                                                   
{'loss': 0.071, 'grad_norm': 0.12901289761066437, 'learning_rate': 9.148619259333955e-05, 'epoch': 4.58}
+
 92%|█████████▏| 687/750 [1:14:31<06:27,  6.15s/it]
 92%|█████████▏| 688/750 [1:14:38<06:22,  6.17s/it]
                                                   
{'loss': 0.075, 'grad_norm': 0.12816192209720612, 'learning_rate': 9.141968135765342e-05, 'epoch': 4.59}
+
 92%|█████████▏| 688/750 [1:14:38<06:22,  6.17s/it]
 92%|█████████▏| 689/750 [1:14:44<06:15,  6.16s/it]
                                                   
{'loss': 0.075, 'grad_norm': 0.13659363985061646, 'learning_rate': 9.135331497405505e-05, 'epoch': 4.59}
+
 92%|█████████▏| 689/750 [1:14:44<06:15,  6.16s/it]
 92%|█████████▏| 690/750 [1:14:50<06:07,  6.12s/it]
                                                   
{'loss': 0.0909, 'grad_norm': 0.1407977193593979, 'learning_rate': 9.128709291752768e-05, 'epoch': 4.6}
+
 92%|█████████▏| 690/750 [1:14:50<06:07,  6.12s/it]
 92%|█████████▏| 691/750 [1:14:56<06:03,  6.17s/it]
                                                   
{'loss': 0.0818, 'grad_norm': 0.13601362705230713, 'learning_rate': 9.122101466571477e-05, 'epoch': 4.61}
+
 92%|█████████▏| 691/750 [1:14:56<06:03,  6.17s/it]
 92%|█████████▏| 692/750 [1:15:02<05:58,  6.18s/it]
                                                   
{'loss': 0.0781, 'grad_norm': 0.13630828261375427, 'learning_rate': 9.115507969890265e-05, 'epoch': 4.61}
+
 92%|█████████▏| 692/750 [1:15:02<05:58,  6.18s/it]
 92%|█████████▏| 693/750 [1:15:08<05:51,  6.16s/it]
                                                   
{'loss': 0.0797, 'grad_norm': 0.1478336751461029, 'learning_rate': 9.108928750000352e-05, 'epoch': 4.62}
+
 92%|█████████▏| 693/750 [1:15:08<05:51,  6.16s/it]
 93%|█████████▎| 694/750 [1:15:14<05:44,  6.15s/it]
                                                   
{'loss': 0.0791, 'grad_norm': 0.1386125087738037, 'learning_rate': 9.102363755453824e-05, 'epoch': 4.63}
+
 93%|█████████▎| 694/750 [1:15:14<05:44,  6.15s/it]
 93%|█████████▎| 695/750 [1:15:21<05:38,  6.15s/it]
                                                   
{'loss': 0.0795, 'grad_norm': 0.1494094729423523, 'learning_rate': 9.095812935061951e-05, 'epoch': 4.63}
+
 93%|█████████▎| 695/750 [1:15:21<05:38,  6.15s/it]
 93%|█████████▎| 696/750 [1:15:27<05:31,  6.14s/it]
                                                   
{'loss': 0.0789, 'grad_norm': 0.15123780071735382, 'learning_rate': 9.089276237893512e-05, 'epoch': 4.64}
+
 93%|█████████▎| 696/750 [1:15:27<05:31,  6.14s/it]
 93%|█████████▎| 697/750 [1:15:33<05:25,  6.14s/it]
                                                   
{'loss': 0.0742, 'grad_norm': 0.145618736743927, 'learning_rate': 9.082753613273131e-05, 'epoch': 4.65}
+
 93%|█████████▎| 697/750 [1:15:33<05:25,  6.14s/it]
 93%|█████████▎| 698/750 [1:15:39<05:19,  6.14s/it]
                                                   
{'loss': 0.0824, 'grad_norm': 0.14783382415771484, 'learning_rate': 9.076245010779625e-05, 'epoch': 4.65}
+
 93%|█████████▎| 698/750 [1:15:39<05:19,  6.14s/it]
 93%|█████████▎| 699/750 [1:15:45<05:13,  6.14s/it]
                                                   
{'loss': 0.0684, 'grad_norm': 0.13855530321598053, 'learning_rate': 9.069750380244364e-05, 'epoch': 4.66}
+
 93%|█████████▎| 699/750 [1:15:45<05:13,  6.14s/it]
 93%|█████████▎| 700/750 [1:15:51<05:07,  6.15s/it]
                                                   
{'loss': 0.0773, 'grad_norm': 0.149196594953537, 'learning_rate': 9.063269671749657e-05, 'epoch': 4.67}
+
 93%|█████████▎| 700/750 [1:15:51<05:07,  6.15s/it]
 93%|█████████▎| 701/750 [1:15:58<05:01,  6.16s/it]
                                                   
{'loss': 0.0713, 'grad_norm': 0.13729305565357208, 'learning_rate': 9.056802835627134e-05, 'epoch': 4.67}
+
 93%|█████████▎| 701/750 [1:15:58<05:01,  6.16s/it]
 94%|█████████▎| 702/750 [1:16:04<04:56,  6.17s/it]
                                                   
{'loss': 0.0803, 'grad_norm': 0.13516099750995636, 'learning_rate': 9.050349822456143e-05, 'epoch': 4.68}
+
 94%|█████████▎| 702/750 [1:16:04<04:56,  6.17s/it]
 94%|█████████▎| 703/750 [1:16:10<04:48,  6.14s/it]
                                                   
{'loss': 0.0737, 'grad_norm': 0.13444063067436218, 'learning_rate': 9.043910583062168e-05, 'epoch': 4.69}
+
 94%|█████████▎| 703/750 [1:16:10<04:48,  6.14s/it]
 94%|█████████▍| 704/750 [1:16:16<04:41,  6.12s/it]
                                                   
{'loss': 0.0886, 'grad_norm': 0.15785068273544312, 'learning_rate': 9.037485068515265e-05, 'epoch': 4.69}
+
 94%|█████████▍| 704/750 [1:16:16<04:41,  6.12s/it]
 94%|█████████▍| 705/750 [1:16:22<04:36,  6.14s/it]
                                                   
{'loss': 0.0837, 'grad_norm': 0.15135006606578827, 'learning_rate': 9.03107323012848e-05, 'epoch': 4.7}
+
 94%|█████████▍| 705/750 [1:16:22<04:36,  6.14s/it]
 94%|█████████▍| 706/750 [1:16:28<04:31,  6.16s/it]
                                                   
{'loss': 0.0861, 'grad_norm': 0.1465722918510437, 'learning_rate': 9.024675019456317e-05, 'epoch': 4.71}
+
 94%|█████████▍| 706/750 [1:16:28<04:31,  6.16s/it]
 94%|█████████▍| 707/750 [1:16:34<04:24,  6.14s/it]
                                                   
{'loss': 0.0778, 'grad_norm': 0.14982181787490845, 'learning_rate': 9.01829038829319e-05, 'epoch': 4.71}
+
 94%|█████████▍| 707/750 [1:16:34<04:24,  6.14s/it]
 94%|█████████▍| 708/750 [1:16:41<04:17,  6.14s/it]
                                                   
{'loss': 0.0672, 'grad_norm': 0.1412889063358307, 'learning_rate': 9.011919288671903e-05, 'epoch': 4.72}
+
 94%|█████████▍| 708/750 [1:16:41<04:17,  6.14s/it]
 95%|█████████▍| 709/750 [1:16:47<04:15,  6.22s/it]
                                                   
{'loss': 0.0822, 'grad_norm': 0.13640610873699188, 'learning_rate': 9.005561672862141e-05, 'epoch': 4.73}
+
 95%|█████████▍| 709/750 [1:16:47<04:15,  6.22s/it]
 95%|█████████▍| 710/750 [1:16:53<04:08,  6.20s/it]
                                                   
{'loss': 0.0761, 'grad_norm': 0.13551074266433716, 'learning_rate': 8.999217493368951e-05, 'epoch': 4.73}
+
 95%|█████████▍| 710/750 [1:16:53<04:08,  6.20s/it]
 95%|█████████▍| 711/750 [1:16:59<04:02,  6.21s/it]
                                                   
{'loss': 0.0865, 'grad_norm': 0.14088967442512512, 'learning_rate': 8.992886702931274e-05, 'epoch': 4.74}
+
 95%|█████████▍| 711/750 [1:16:59<04:02,  6.21s/it]
 95%|█████████▍| 712/750 [1:17:06<03:56,  6.22s/it]
                                                   
{'loss': 0.0772, 'grad_norm': 0.14193542301654816, 'learning_rate': 8.986569254520445e-05, 'epoch': 4.75}
+
 95%|█████████▍| 712/750 [1:17:06<03:56,  6.22s/it]
 95%|█████████▌| 713/750 [1:17:12<03:49,  6.21s/it]
                                                   
{'loss': 0.086, 'grad_norm': 0.13961544632911682, 'learning_rate': 8.980265101338746e-05, 'epoch': 4.75}
+
 95%|█████████▌| 713/750 [1:17:12<03:49,  6.21s/it]
 95%|█████████▌| 714/750 [1:17:18<03:43,  6.19s/it]
                                                   
{'loss': 0.0719, 'grad_norm': 0.1363149732351303, 'learning_rate': 8.973974196817932e-05, 'epoch': 4.76}
+
 95%|█████████▌| 714/750 [1:17:18<03:43,  6.19s/it]
 95%|█████████▌| 715/750 [1:17:24<03:36,  6.18s/it]
                                                   
{'loss': 0.077, 'grad_norm': 0.14475861191749573, 'learning_rate': 8.967696494617803e-05, 'epoch': 4.77}
+
 95%|█████████▌| 715/750 [1:17:24<03:36,  6.18s/it]
 95%|█████████▌| 716/750 [1:17:30<03:29,  6.17s/it]
                                                   
{'loss': 0.0785, 'grad_norm': 0.13795825839042664, 'learning_rate': 8.961431948624752e-05, 'epoch': 4.77}
+
 95%|█████████▌| 716/750 [1:17:30<03:29,  6.17s/it]
 96%|█████████▌| 717/750 [1:17:36<03:23,  6.18s/it]
                                                   
{'loss': 0.0764, 'grad_norm': 0.1390857696533203, 'learning_rate': 8.955180512950356e-05, 'epoch': 4.78}
+
 96%|█████████▌| 717/750 [1:17:36<03:23,  6.18s/it]
 96%|█████████▌| 718/750 [1:17:43<03:18,  6.19s/it]
                                                   
{'loss': 0.0703, 'grad_norm': 0.12633129954338074, 'learning_rate': 8.948942141929963e-05, 'epoch': 4.79}
+
 96%|█████████▌| 718/750 [1:17:43<03:18,  6.19s/it]
 96%|█████████▌| 719/750 [1:17:49<03:11,  6.19s/it]
                                                   
{'loss': 0.0777, 'grad_norm': 0.13622745871543884, 'learning_rate': 8.942716790121283e-05, 'epoch': 4.79}
+
 96%|█████████▌| 719/750 [1:17:49<03:11,  6.19s/it]
 96%|█████████▌| 720/750 [1:17:55<03:06,  6.20s/it]
                                                   
{'loss': 0.0756, 'grad_norm': 0.134665846824646, 'learning_rate': 8.936504412303008e-05, 'epoch': 4.8}
+
 96%|█████████▌| 720/750 [1:17:55<03:06,  6.20s/it]
 96%|█████████▌| 721/750 [1:18:01<02:59,  6.20s/it]
                                                   
{'loss': 0.086, 'grad_norm': 0.14154991507530212, 'learning_rate': 8.93030496347342e-05, 'epoch': 4.81}
+
 96%|█████████▌| 721/750 [1:18:01<02:59,  6.20s/it]
 96%|█████████▋| 722/750 [1:18:07<02:53,  6.19s/it]
                                                   
{'loss': 0.0827, 'grad_norm': 0.1462387591600418, 'learning_rate': 8.924118398849037e-05, 'epoch': 4.81}
+
 96%|█████████▋| 722/750 [1:18:07<02:53,  6.19s/it]
 96%|█████████▋| 723/750 [1:18:14<02:46,  6.17s/it]
                                                   
{'loss': 0.0938, 'grad_norm': 0.14661766588687897, 'learning_rate': 8.917944673863242e-05, 'epoch': 4.82}
+
 96%|█████████▋| 723/750 [1:18:14<02:46,  6.17s/it]
 97%|█████████▋| 724/750 [1:18:20<02:39,  6.15s/it]
                                                   
{'loss': 0.0765, 'grad_norm': 0.14469976723194122, 'learning_rate': 8.911783744164937e-05, 'epoch': 4.83}
+
 97%|█████████▋| 724/750 [1:18:20<02:39,  6.15s/it]
 97%|█████████▋| 725/750 [1:18:26<02:33,  6.14s/it]
                                                   
{'loss': 0.0743, 'grad_norm': 0.1378207802772522, 'learning_rate': 8.905635565617214e-05, 'epoch': 4.83}
+
 97%|█████████▋| 725/750 [1:18:26<02:33,  6.14s/it]
 97%|█████████▋| 726/750 [1:18:32<02:26,  6.12s/it]
                                                   
{'loss': 0.0795, 'grad_norm': 0.14783820509910583, 'learning_rate': 8.899500094296009e-05, 'epoch': 4.84}
+
 97%|█████████▋| 726/750 [1:18:32<02:26,  6.12s/it]
 97%|█████████▋| 727/750 [1:18:38<02:21,  6.14s/it]
                                                   
{'loss': 0.0808, 'grad_norm': 0.14286793768405914, 'learning_rate': 8.893377286488804e-05, 'epoch': 4.85}
+
 97%|█████████▋| 727/750 [1:18:38<02:21,  6.14s/it]
 97%|█████████▋| 728/750 [1:18:44<02:16,  6.20s/it]
                                                   
{'loss': 0.0775, 'grad_norm': 0.1347174197435379, 'learning_rate': 8.887267098693303e-05, 'epoch': 4.85}
+
 97%|█████████▋| 728/750 [1:18:44<02:16,  6.20s/it]
 97%|█████████▋| 729/750 [1:18:50<02:09,  6.17s/it]
                                                   
{'loss': 0.0803, 'grad_norm': 0.14919400215148926, 'learning_rate': 8.881169487616148e-05, 'epoch': 4.86}
+
 97%|█████████▋| 729/750 [1:18:50<02:09,  6.17s/it]
 97%|█████████▋| 730/750 [1:18:57<02:03,  6.16s/it]
                                                   
{'loss': 0.0687, 'grad_norm': 0.15091973543167114, 'learning_rate': 8.875084410171615e-05, 'epoch': 4.87}
+
 97%|█████████▋| 730/750 [1:18:57<02:03,  6.16s/it]
 97%|█████████▋| 731/750 [1:19:03<01:58,  6.21s/it]
                                                   
{'loss': 0.0862, 'grad_norm': 0.14625585079193115, 'learning_rate': 8.869011823480348e-05, 'epoch': 4.87}
+
 97%|█████████▋| 731/750 [1:19:03<01:58,  6.21s/it]
 98%|█████████▊| 732/750 [1:19:09<01:51,  6.17s/it]
                                                   
{'loss': 0.083, 'grad_norm': 0.14147014915943146, 'learning_rate': 8.862951684868085e-05, 'epoch': 4.88}
+
 98%|█████████▊| 732/750 [1:19:09<01:51,  6.17s/it]
 98%|█████████▊| 733/750 [1:19:15<01:44,  6.17s/it]
                                                   
{'loss': 0.0778, 'grad_norm': 0.14377067983150482, 'learning_rate': 8.856903951864397e-05, 'epoch': 4.89}
+
 98%|█████████▊| 733/750 [1:19:15<01:44,  6.17s/it]
 98%|█████████▊| 734/750 [1:19:21<01:39,  6.20s/it]
                                                   
{'loss': 0.0792, 'grad_norm': 0.13911950588226318, 'learning_rate': 8.85086858220144e-05, 'epoch': 4.89}
+
 98%|█████████▊| 734/750 [1:19:21<01:39,  6.20s/it]
 98%|█████████▊| 735/750 [1:19:28<01:32,  6.18s/it]
                                                   
{'loss': 0.0802, 'grad_norm': 0.14028075337409973, 'learning_rate': 8.844845533812719e-05, 'epoch': 4.9}
+
 98%|█████████▊| 735/750 [1:19:28<01:32,  6.18s/it]
 98%|█████████▊| 736/750 [1:19:34<01:26,  6.16s/it]
                                                   
{'loss': 0.0812, 'grad_norm': 0.14348360896110535, 'learning_rate': 8.838834764831844e-05, 'epoch': 4.91}
+
 98%|█████████▊| 736/750 [1:19:34<01:26,  6.16s/it]
 98%|█████████▊| 737/750 [1:19:40<01:19,  6.14s/it]
                                                   
{'loss': 0.0809, 'grad_norm': 0.13959698379039764, 'learning_rate': 8.83283623359132e-05, 'epoch': 4.91}
+
 98%|█████████▊| 737/750 [1:19:40<01:19,  6.14s/it]
 98%|█████████▊| 738/750 [1:19:46<01:13,  6.14s/it]
                                                   
{'loss': 0.0701, 'grad_norm': 0.1444099247455597, 'learning_rate': 8.826849898621327e-05, 'epoch': 4.92}
+
 98%|█████████▊| 738/750 [1:19:46<01:13,  6.14s/it]
 99%|█████████▊| 739/750 [1:19:52<01:07,  6.13s/it]
                                                   
{'loss': 0.0744, 'grad_norm': 0.1581725776195526, 'learning_rate': 8.82087571864852e-05, 'epoch': 4.93}
+
 99%|█████████▊| 739/750 [1:19:52<01:07,  6.13s/it]
 99%|█████████▊| 740/750 [1:19:58<01:01,  6.14s/it]
                                                   
{'loss': 0.0797, 'grad_norm': 0.14414192736148834, 'learning_rate': 8.814913652594829e-05, 'epoch': 4.93}
+
 99%|█████████▊| 740/750 [1:19:58<01:01,  6.14s/it]
 99%|█████████▉| 741/750 [1:20:04<00:55,  6.12s/it]
                                                   
{'loss': 0.0913, 'grad_norm': 0.1519823670387268, 'learning_rate': 8.808963659576277e-05, 'epoch': 4.94}
+
 99%|█████████▉| 741/750 [1:20:04<00:55,  6.12s/it]
 99%|█████████▉| 742/750 [1:20:10<00:49,  6.13s/it]
                                                   
{'loss': 0.0901, 'grad_norm': 0.15351378917694092, 'learning_rate': 8.803025698901805e-05, 'epoch': 4.95}
+
 99%|█████████▉| 742/750 [1:20:10<00:49,  6.13s/it]
 99%|█████████▉| 743/750 [1:20:16<00:42,  6.12s/it]
                                                   
{'loss': 0.0753, 'grad_norm': 0.1508694738149643, 'learning_rate': 8.797099730072091e-05, 'epoch': 4.95}
+
 99%|█████████▉| 743/750 [1:20:16<00:42,  6.12s/it]
 99%|█████████▉| 744/750 [1:20:23<00:36,  6.13s/it]
                                                   
{'loss': 0.0758, 'grad_norm': 0.13677740097045898, 'learning_rate': 8.791185712778405e-05, 'epoch': 4.96}
+
 99%|█████████▉| 744/750 [1:20:23<00:36,  6.13s/it]
 99%|█████████▉| 745/750 [1:20:29<00:30,  6.10s/it]
                                                   
{'loss': 0.0804, 'grad_norm': 0.1491473764181137, 'learning_rate': 8.785283606901446e-05, 'epoch': 4.97}
+
 99%|█████████▉| 745/750 [1:20:29<00:30,  6.10s/it]
 99%|█████████▉| 746/750 [1:20:35<00:24,  6.08s/it]
                                                   
{'loss': 0.094, 'grad_norm': 0.15220019221305847, 'learning_rate': 8.779393372510207e-05, 'epoch': 4.97}
+
 99%|█████████▉| 746/750 [1:20:35<00:24,  6.08s/it]
100%|█████████▉| 747/750 [1:20:41<00:18,  6.13s/it]
                                                   
{'loss': 0.0762, 'grad_norm': 0.14122961461544037, 'learning_rate': 8.773514969860834e-05, 'epoch': 4.98}
+
100%|█████████▉| 747/750 [1:20:41<00:18,  6.13s/it]
100%|█████████▉| 748/750 [1:20:47<00:12,  6.14s/it]
                                                   
{'loss': 0.0837, 'grad_norm': 0.14642593264579773, 'learning_rate': 8.767648359395506e-05, 'epoch': 4.99}
+
100%|█████████▉| 748/750 [1:20:47<00:12,  6.14s/it]
100%|█████████▉| 749/750 [1:20:53<00:06,  6.16s/it]
                                                   
{'loss': 0.0769, 'grad_norm': 0.1552942991256714, 'learning_rate': 8.761793501741308e-05, 'epoch': 4.99}
+
100%|█████████▉| 749/750 [1:20:53<00:06,  6.16s/it]
100%|██████████| 750/750 [1:21:02<00:00,  6.82s/it]
                                                   
{'loss': 0.0868, 'grad_norm': 0.1520676612854004, 'learning_rate': 8.755950357709131e-05, 'epoch': 5.0}
+
100%|██████████| 750/750 [1:21:02<00:00,  6.82s/it][INFO|trainer.py:3515] 2025-06-26 23:37:08,959 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-750
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8c4d-18d6f35e09b55252103317af;ee7edb67-b044-44d2-91e4-0dd75ae3e79d)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-26 23:37:09,353 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-750/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-26 23:37:09,353 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-750/special_tokens_map.json
+[2025-06-26 23:37:10,576] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Checkpoint global_step750 is begin to save!
+[2025-06-26 23:37:10,599] [INFO] [logging.py:107:log_dist] [Rank 0] Saving model checkpoint: ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-750/global_step750/mp_rank_00_model_states.pt
+[INFO|trainer.py:3607] 2025-06-26 23:37:10,729 >> Deleting older checkpoint [outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-600] due to args.save_total_limit
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8c4e-4e6488365754e3f063daf878;bcf46ae6-b767-4cc0-becc-b6f83a3cd26e)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8c4e-2f6c08887a7f7703090a08db;ddb87f63-cbf7-4e72-9f28-6957364d3759)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8c4e-29d9969d4124394d1299f63f;01d5e0d2-1603-4e08-8daa-22e8baf88c25)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8c4e-55fb03ef2ef563de53037d0a;c5804b38-2a76-482b-8e81-0206a4f1c9e4)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+[INFO|trainer.py:2406] 2025-06-26 23:37:11,096 >> 
+
+Training completed. Do not forget to share your model on huggingface.co/models =)
+
+
+[INFO|trainer.py:2644] 2025-06-26 23:37:11,154 >> Loading best model from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-400 (score: 0.1427709013223648).
+[INFO|deepspeed.py:431] 2025-06-26 23:37:11,155 >> Attempting to resume from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-400
+[2025-06-26 23:37:11,156] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Begin Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt...
+[2025-06-26 23:37:11,175] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] End Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt...
+[2025-06-26 23:37:11,176] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Begin Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt...
+[2025-06-26 23:37:11,191] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] End Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-400/global_step400/mp_rank_00_model_states.pt...
+[2025-06-26 23:37:11,224] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] Begin Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt...
+[2025-06-26 23:37:11,243] [INFO] [logging.py:107:log_dist] [Rank 0] [Torch] End Load checkpoint from ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-400/global_step400/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt...
+[2025-06-26 23:37:11,243] [INFO] [engine.py:3277:_get_all_zero_checkpoint_state_dicts] successfully read 4 ZeRO state_dicts for rank 0
+[2025-06-26 23:37:11,251] [INFO] [engine.py:3227:_load_zero_checkpoint] loading 4 zero partition checkpoints for rank 0
+
                                                   
{'train_runtime': 4880.0924, 'train_samples_per_second': 15.369, 'train_steps_per_second': 0.154, 'train_loss': 0.11640988408029079, 'epoch': 5.0}
+
100%|██████████| 750/750 [1:21:18<00:00,  6.82s/it][INFO|trainer.py:2447] 2025-06-26 23:37:11,254 >> Deleting older checkpoint [outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/checkpoint-750] due to args.save_total_limit
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8c4f-0cafd14f1add9fe46988ac89;9388118b-a815-42e5-b22d-56ef92cc6311)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8c4f-616698264ce1d8ea0518023b;4f4bbec2-03d6-4286-827a-e7801dd01c23)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8c4f-78e845c42535d886650a47db;c93a6706-d413-49d5-a92a-c2bf27b61b9b)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8c4f-7b56338d15df9fa86515d876;48782ba2-15c4-4a90-b3bb-643a4f09f6bb)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+
100%|██████████| 750/750 [1:21:18<00:00,  6.50s/it]
+[INFO|trainer.py:3515] 2025-06-26 23:37:24,829 >> Saving model checkpoint to ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/
+/home/iitm_admin/miniconda3/envs/midalign/lib/python3.9/site-packages/peft/utils/other.py:1110: UserWarning: Unable to fetch remote file due to the following error 403 Client Error. (Request ID: Root=1-685d8c5c-6c9a91492c8a8c8655107185;c8d951e0-6741-49a2-845c-8b804ff3e640)
+
+Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/resolve/main/config.json.
+Access to model meta-llama/Llama-3.1-8B-Instruct is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct to ask for access. - silently ignoring the lookup for the file config.json in meta-llama/Llama-3.1-8B-Instruct.
+  warnings.warn(
+[INFO|tokenization_utils_base.py:2684] 2025-06-26 23:37:25,146 >> tokenizer config file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/tokenizer_config.json
+[INFO|tokenization_utils_base.py:2693] 2025-06-26 23:37:25,147 >> Special tokens file saved in ./outputs/data/reason/meta-llama/Llama-3.1-8B-Instruct/te/baseline/data_15000_1000/special_tokens_map.json
+***** train metrics *****
+  epoch                    =          5.0
+  total_flos               = 6458914757GF
+  train_loss               =       0.1164
+  train_runtime            =   1:21:20.09
+  train_samples            =        15000
+  train_samples_per_second =       15.369
+  train_steps_per_second   =        0.154
+06/26/2025 23:37:26 - INFO - __main__ - *** Evaluate ***
+[INFO|trainer.py:3831] 2025-06-26 23:37:26,489 >> 
+***** Running Evaluation *****
+[INFO|trainer.py:3833] 2025-06-26 23:37:26,490 >>   Num examples = 1000
+[INFO|trainer.py:3836] 2025-06-26 23:37:26,490 >>   Batch size = 25
+
  0%|          | 0/10 [00:00<?, ?it/s]
 20%|██        | 2/10 [00:01<00:07,  1.08it/s]
 30%|███       | 3/10 [00:03<00:09,  1.32s/it]
 40%|████      | 4/10 [00:05<00:09,  1.52s/it]
 50%|█████     | 5/10 [00:07<00:08,  1.64s/it]
 60%|██████    | 6/10 [00:09<00:06,  1.70s/it]
 70%|███████   | 7/10 [00:11<00:05,  1.75s/it]
 80%|████████  | 8/10 [00:12<00:03,  1.77s/it]
 90%|█████████ | 9/10 [00:14<00:01,  1.80s/it]
100%|██████████| 10/10 [00:18<00:00,  2.34s/it]
100%|██████████| 10/10 [00:24<00:00,  2.40s/it]
+***** eval metrics *****
+  epoch                   =        5.0
+  eval_accuracy           =     0.2946
+  eval_loss               =     0.1428
+  eval_runtime            = 0:00:34.30
+  eval_samples            =       1000
+  eval_samples_per_second =     29.152
+  eval_steps_per_second   =      0.292
+  perplexity              =     1.1535
+[INFO|modelcard.py:449] 2025-06-26 23:38:01,023 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.2945705911089399}]}
diff --git a/te/baseline/data_15000_1000/train_results.json b/te/baseline/data_15000_1000/train_results.json
new file mode 100644
index 0000000000000000000000000000000000000000..979f145b4c43eeeaa43b1312d69bc5517ea48372
--- /dev/null
+++ b/te/baseline/data_15000_1000/train_results.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe547aab550107e5da34ba929469978d33bed50544d9a4d3fc2a532eb360cb5d
+size 237
diff --git a/te/baseline/data_15000_1000/trainer_state.json b/te/baseline/data_15000_1000/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..b47d889ed4d16ad1b548bc2e462eb6137fc6f5a6
--- /dev/null
+++ b/te/baseline/data_15000_1000/trainer_state.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4aab812c294e2983165caec2634be41894eb584e7791a8bfe3b5134037860fbe
+size 129406
diff --git a/te/baseline/data_15000_1000/training_args.bin b/te/baseline/data_15000_1000/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..48852a5b49fabd0bc05f759b98bc85ab6d2de0cc
--- /dev/null
+++ b/te/baseline/data_15000_1000/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb70f1f7eb340def6adc18e2dbd35ca24fe7cef2fe7c9151a297ff1bb9ed2a09
+size 7761