Auto fine-tune: 16589 examples (LoRA adapter)

adapter_config.json

@@ -32,8 +32,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "query_proj",
-    "value_proj"
+    "value_proj",
+    "query_proj"
   ],
   "target_parameters": null,
   "task_type": "SEQ_CLS",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:52b107fcc83cba29c7d232be9a0543f09a8f49c7a4c859dd8a2bfe6769db13d3
+oid sha256:2259511c30404871994c04053dc32ece5c228c38d3ad66dcfedeb0d7c8f9e932
 size 9450704

checkpoint-10075/README.md

@@ -0,0 +1,206 @@
+---
+base_model: protectai/deberta-v3-base-prompt-injection-v2
+library_name: peft
+tags:
+- base_model:adapter:protectai/deberta-v3-base-prompt-injection-v2
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

checkpoint-10075/adapter_config.json

@@ -0,0 +1,44 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "protectai/deberta-v3-base-prompt-injection-v2",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "value_proj",
+    "query_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-10075/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2259511c30404871994c04053dc32ece5c228c38d3ad66dcfedeb0d7c8f9e932
+size 9450704

checkpoint-10075/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8846c7d1e2bbe86e65ccdbea3cdfe8ba50fea5cc1cc8390acc56285da563631f
+size 18930123

checkpoint-10075/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f895cb24f053aa26c8b41208283871b8a5402ef60ef88e745c7bbe5a4d9513f
+size 14645

checkpoint-10075/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e345a8c23dc269cbd2e6213ec8367d7085363bf7a3b8452873f9ad640b247f2
+size 1383

checkpoint-10075/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afcc4a2e8f660dc98928734873f7ffb0f1923111fbbe92df7c70edba32c97e70
+size 1465

checkpoint-10075/special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-10075/tokenizer_config.json

@@ -0,0 +1,63 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128000": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "split_by_punct": false,
+  "stride": 0,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}

checkpoint-10075/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:859efea2201c1469208153e54e3977a99dc6a369e5e443e74e5277d4b63a6e19
+size 5841

checkpoint-2015/README.md

@@ -0,0 +1,206 @@
+---
+base_model: protectai/deberta-v3-base-prompt-injection-v2
+library_name: peft
+tags:
+- base_model:adapter:protectai/deberta-v3-base-prompt-injection-v2
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

checkpoint-2015/adapter_config.json

@@ -0,0 +1,44 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "protectai/deberta-v3-base-prompt-injection-v2",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "value_proj",
+    "query_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-2015/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f79c088a4a69a85eae07f72308293014f00d3d78ece33ebd94b2e3904287747b
+size 9450704

checkpoint-2015/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10d30c933d75c288961adf2a4c9483a799c2e8ef4c09ff28e91f49fcb329585f
+size 18930123

checkpoint-2015/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d94b3980e6f308a253737a437b363ae6dffd43f5f4bf0d8e348527be5e3b8539
+size 14645

checkpoint-2015/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1dc6f823943d1c2dab64f49a85bfe23d24a5a6c15134dcffcab60ce7d6e9cefe
+size 1383

checkpoint-2015/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04a0a96278289385313616ae07a6f65745b9bf235e4be5e924824981c8403c59
+size 1465

checkpoint-2015/special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-2015/tokenizer_config.json

@@ -0,0 +1,63 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128000": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "split_by_punct": false,
+  "stride": 0,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}

checkpoint-2015/trainer_state.json

@@ -0,0 +1,1441 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 2015,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.004962779156327543,
+      "grad_norm": NaN,
+      "learning_rate": 1.9982133995037223e-05,
+      "loss": 4.0715,
+      "step": 10
+    },
+    {
+      "epoch": 0.009925558312655087,
+      "grad_norm": 7.404272079467773,
+      "learning_rate": 1.996228287841191e-05,
+      "loss": 2.8811,
+      "step": 20
+    },
+    {
+      "epoch": 0.01488833746898263,
+      "grad_norm": 13.565997123718262,
+      "learning_rate": 1.99424317617866e-05,
+      "loss": 3.1689,
+      "step": 30
+    },
+    {
+      "epoch": 0.019851116625310174,
+      "grad_norm": 15.281591415405273,
+      "learning_rate": 1.9922580645161292e-05,
+      "loss": 3.0516,
+      "step": 40
+    },
+    {
+      "epoch": 0.02481389578163772,
+      "grad_norm": 7.509647369384766,
+      "learning_rate": 1.9902729528535983e-05,
+      "loss": 2.9276,
+      "step": 50
+    },
+    {
+      "epoch": 0.02977667493796526,
+      "grad_norm": 7.238119602203369,
+      "learning_rate": 1.988287841191067e-05,
+      "loss": 2.0641,
+      "step": 60
+    },
+    {
+      "epoch": 0.034739454094292806,
+      "grad_norm": 8.467841148376465,
+      "learning_rate": 1.9863027295285362e-05,
+      "loss": 1.8259,
+      "step": 70
+    },
+    {
+      "epoch": 0.03970223325062035,
+      "grad_norm": 7.360463619232178,
+      "learning_rate": 1.984317617866005e-05,
+      "loss": 1.7238,
+      "step": 80
+    },
+    {
+      "epoch": 0.04466501240694789,
+      "grad_norm": 6.133955478668213,
+      "learning_rate": 1.982332506203474e-05,
+      "loss": 1.3318,
+      "step": 90
+    },
+    {
+      "epoch": 0.04962779156327544,
+      "grad_norm": 6.2159295082092285,
+      "learning_rate": 1.980347394540943e-05,
+      "loss": 1.1172,
+      "step": 100
+    },
+    {
+      "epoch": 0.05459057071960298,
+      "grad_norm": 2.4206221103668213,
+      "learning_rate": 1.9783622828784122e-05,
+      "loss": 1.033,
+      "step": 110
+    },
+    {
+      "epoch": 0.05955334987593052,
+      "grad_norm": 5.191859245300293,
+      "learning_rate": 1.976377171215881e-05,
+      "loss": 0.98,
+      "step": 120
+    },
+    {
+      "epoch": 0.06451612903225806,
+      "grad_norm": 2.742246150970459,
+      "learning_rate": 1.97439205955335e-05,
+      "loss": 0.7428,
+      "step": 130
+    },
+    {
+      "epoch": 0.06947890818858561,
+      "grad_norm": 4.614717960357666,
+      "learning_rate": 1.972406947890819e-05,
+      "loss": 0.9041,
+      "step": 140
+    },
+    {
+      "epoch": 0.07444168734491315,
+      "grad_norm": 3.7935266494750977,
+      "learning_rate": 1.9704218362282882e-05,
+      "loss": 0.5919,
+      "step": 150
+    },
+    {
+      "epoch": 0.0794044665012407,
+      "grad_norm": 3.5459160804748535,
+      "learning_rate": 1.968436724565757e-05,
+      "loss": 0.7932,
+      "step": 160
+    },
+    {
+      "epoch": 0.08436724565756824,
+      "grad_norm": 1.8404773473739624,
+      "learning_rate": 1.966451612903226e-05,
+      "loss": 0.7619,
+      "step": 170
+    },
+    {
+      "epoch": 0.08933002481389578,
+      "grad_norm": 2.2088851928710938,
+      "learning_rate": 1.964466501240695e-05,
+      "loss": 0.6204,
+      "step": 180
+    },
+    {
+      "epoch": 0.09429280397022333,
+      "grad_norm": 1.566972017288208,
+      "learning_rate": 1.962481389578164e-05,
+      "loss": 0.7081,
+      "step": 190
+    },
+    {
+      "epoch": 0.09925558312655088,
+      "grad_norm": 1.225921630859375,
+      "learning_rate": 1.960496277915633e-05,
+      "loss": 0.5577,
+      "step": 200
+    },
+    {
+      "epoch": 0.10421836228287841,
+      "grad_norm": 2.0241658687591553,
+      "learning_rate": 1.958511166253102e-05,
+      "loss": 0.5101,
+      "step": 210
+    },
+    {
+      "epoch": 0.10918114143920596,
+      "grad_norm": 2.1363158226013184,
+      "learning_rate": 1.956526054590571e-05,
+      "loss": 0.4952,
+      "step": 220
+    },
+    {
+      "epoch": 0.1141439205955335,
+      "grad_norm": 2.7994675636291504,
+      "learning_rate": 1.9545409429280396e-05,
+      "loss": 0.5103,
+      "step": 230
+    },
+    {
+      "epoch": 0.11910669975186104,
+      "grad_norm": 2.4477338790893555,
+      "learning_rate": 1.9525558312655087e-05,
+      "loss": 0.3801,
+      "step": 240
+    },
+    {
+      "epoch": 0.12406947890818859,
+      "grad_norm": 2.0796492099761963,
+      "learning_rate": 1.9505707196029778e-05,
+      "loss": 0.4258,
+      "step": 250
+    },
+    {
+      "epoch": 0.12903225806451613,
+      "grad_norm": 1.3138341903686523,
+      "learning_rate": 1.948585607940447e-05,
+      "loss": 0.4838,
+      "step": 260
+    },
+    {
+      "epoch": 0.13399503722084366,
+      "grad_norm": 1.5272380113601685,
+      "learning_rate": 1.9466004962779156e-05,
+      "loss": 0.366,
+      "step": 270
+    },
+    {
+      "epoch": 0.13895781637717122,
+      "grad_norm": 3.6165964603424072,
+      "learning_rate": 1.9446153846153847e-05,
+      "loss": 0.4171,
+      "step": 280
+    },
+    {
+      "epoch": 0.14392059553349876,
+      "grad_norm": 2.2593328952789307,
+      "learning_rate": 1.9426302729528538e-05,
+      "loss": 0.3028,
+      "step": 290
+    },
+    {
+      "epoch": 0.1488833746898263,
+      "grad_norm": 1.5910056829452515,
+      "learning_rate": 1.940645161290323e-05,
+      "loss": 0.3047,
+      "step": 300
+    },
+    {
+      "epoch": 0.15384615384615385,
+      "grad_norm": 1.767904281616211,
+      "learning_rate": 1.9386600496277917e-05,
+      "loss": 0.4552,
+      "step": 310
+    },
+    {
+      "epoch": 0.1588089330024814,
+      "grad_norm": 1.7085593938827515,
+      "learning_rate": 1.9366749379652608e-05,
+      "loss": 0.2556,
+      "step": 320
+    },
+    {
+      "epoch": 0.16377171215880892,
+      "grad_norm": 1.355582594871521,
+      "learning_rate": 1.9346898263027295e-05,
+      "loss": 0.4337,
+      "step": 330
+    },
+    {
+      "epoch": 0.1687344913151365,
+      "grad_norm": 0.8261783123016357,
+      "learning_rate": 1.9327047146401986e-05,
+      "loss": 0.3328,
+      "step": 340
+    },
+    {
+      "epoch": 0.17369727047146402,
+      "grad_norm": 0.8605831861495972,
+      "learning_rate": 1.9307196029776677e-05,
+      "loss": 0.2587,
+      "step": 350
+    },
+    {
+      "epoch": 0.17866004962779156,
+      "grad_norm": 2.7887861728668213,
+      "learning_rate": 1.9287344913151368e-05,
+      "loss": 0.3559,
+      "step": 360
+    },
+    {
+      "epoch": 0.18362282878411912,
+      "grad_norm": 1.0830411911010742,
+      "learning_rate": 1.9267493796526055e-05,
+      "loss": 0.1826,
+      "step": 370
+    },
+    {
+      "epoch": 0.18858560794044665,
+      "grad_norm": 3.8424558639526367,
+      "learning_rate": 1.9247642679900746e-05,
+      "loss": 0.2587,
+      "step": 380
+    },
+    {
+      "epoch": 0.1935483870967742,
+      "grad_norm": 3.894951343536377,
+      "learning_rate": 1.9227791563275434e-05,
+      "loss": 0.2034,
+      "step": 390
+    },
+    {
+      "epoch": 0.19851116625310175,
+      "grad_norm": 3.139410972595215,
+      "learning_rate": 1.9207940446650125e-05,
+      "loss": 0.2387,
+      "step": 400
+    },
+    {
+      "epoch": 0.20347394540942929,
+      "grad_norm": 1.7462635040283203,
+      "learning_rate": 1.9188089330024816e-05,
+      "loss": 0.2626,
+      "step": 410
+    },
+    {
+      "epoch": 0.20843672456575682,
+      "grad_norm": 4.002986907958984,
+      "learning_rate": 1.9168238213399507e-05,
+      "loss": 0.1782,
+      "step": 420
+    },
+    {
+      "epoch": 0.21339950372208435,
+      "grad_norm": 2.251654624938965,
+      "learning_rate": 1.9148387096774194e-05,
+      "loss": 0.1621,
+      "step": 430
+    },
+    {
+      "epoch": 0.21836228287841192,
+      "grad_norm": 0.6951057314872742,
+      "learning_rate": 1.9128535980148885e-05,
+      "loss": 0.1937,
+      "step": 440
+    },
+    {
+      "epoch": 0.22332506203473945,
+      "grad_norm": 0.8540646433830261,
+      "learning_rate": 1.9108684863523576e-05,
+      "loss": 0.1913,
+      "step": 450
+    },
+    {
+      "epoch": 0.228287841191067,
+      "grad_norm": 0.9929500222206116,
+      "learning_rate": 1.9088833746898267e-05,
+      "loss": 0.2433,
+      "step": 460
+    },
+    {
+      "epoch": 0.23325062034739455,
+      "grad_norm": 3.8289272785186768,
+      "learning_rate": 1.9068982630272954e-05,
+      "loss": 0.1903,
+      "step": 470
+    },
+    {
+      "epoch": 0.23821339950372208,
+      "grad_norm": 4.537591934204102,
+      "learning_rate": 1.9049131513647645e-05,
+      "loss": 0.2136,
+      "step": 480
+    },
+    {
+      "epoch": 0.24317617866004962,
+      "grad_norm": 0.20700696110725403,
+      "learning_rate": 1.9029280397022333e-05,
+      "loss": 0.1888,
+      "step": 490
+    },
+    {
+      "epoch": 0.24813895781637718,
+      "grad_norm": 0.4332411289215088,
+      "learning_rate": 1.9009429280397024e-05,
+      "loss": 0.2358,
+      "step": 500
+    },
+    {
+      "epoch": 0.2531017369727047,
+      "grad_norm": 1.050528645515442,
+      "learning_rate": 1.8989578163771715e-05,
+      "loss": 0.0832,
+      "step": 510
+    },
+    {
+      "epoch": 0.25806451612903225,
+      "grad_norm": 0.22720620036125183,
+      "learning_rate": 1.8969727047146406e-05,
+      "loss": 0.1272,
+      "step": 520
+    },
+    {
+      "epoch": 0.2630272952853598,
+      "grad_norm": 3.576136350631714,
+      "learning_rate": 1.8949875930521093e-05,
+      "loss": 0.2066,
+      "step": 530
+    },
+    {
+      "epoch": 0.2679900744416873,
+      "grad_norm": 0.33265048265457153,
+      "learning_rate": 1.8930024813895784e-05,
+      "loss": 0.0556,
+      "step": 540
+    },
+    {
+      "epoch": 0.2729528535980149,
+      "grad_norm": 0.6722800731658936,
+      "learning_rate": 1.891017369727047e-05,
+      "loss": 0.0807,
+      "step": 550
+    },
+    {
+      "epoch": 0.27791563275434245,
+      "grad_norm": 1.2628779411315918,
+      "learning_rate": 1.8890322580645163e-05,
+      "loss": 0.24,
+      "step": 560
+    },
+    {
+      "epoch": 0.28287841191067,
+      "grad_norm": 0.27271902561187744,
+      "learning_rate": 1.8870471464019853e-05,
+      "loss": 0.077,
+      "step": 570
+    },
+    {
+      "epoch": 0.2878411910669975,
+      "grad_norm": 3.738210916519165,
+      "learning_rate": 1.885062034739454e-05,
+      "loss": 0.395,
+      "step": 580
+    },
+    {
+      "epoch": 0.29280397022332505,
+      "grad_norm": 0.6131235361099243,
+      "learning_rate": 1.8830769230769232e-05,
+      "loss": 0.1971,
+      "step": 590
+    },
+    {
+      "epoch": 0.2977667493796526,
+      "grad_norm": 0.8646567463874817,
+      "learning_rate": 1.8810918114143923e-05,
+      "loss": 0.0669,
+      "step": 600
+    },
+    {
+      "epoch": 0.3027295285359802,
+      "grad_norm": 4.964555263519287,
+      "learning_rate": 1.8791066997518614e-05,
+      "loss": 0.2324,
+      "step": 610
+    },
+    {
+      "epoch": 0.3076923076923077,
+      "grad_norm": 0.5028713941574097,
+      "learning_rate": 1.87712158808933e-05,
+      "loss": 0.1819,
+      "step": 620
+    },
+    {
+      "epoch": 0.31265508684863524,
+      "grad_norm": 0.31731173396110535,
+      "learning_rate": 1.8751364764267992e-05,
+      "loss": 0.1463,
+      "step": 630
+    },
+    {
+      "epoch": 0.3176178660049628,
+      "grad_norm": 0.23767873644828796,
+      "learning_rate": 1.873151364764268e-05,
+      "loss": 0.0874,
+      "step": 640
+    },
+    {
+      "epoch": 0.3225806451612903,
+      "grad_norm": 0.5757867693901062,
+      "learning_rate": 1.871166253101737e-05,
+      "loss": 0.1267,
+      "step": 650
+    },
+    {
+      "epoch": 0.32754342431761785,
+      "grad_norm": 2.5683581829071045,
+      "learning_rate": 1.869181141439206e-05,
+      "loss": 0.0823,
+      "step": 660
+    },
+    {
+      "epoch": 0.3325062034739454,
+      "grad_norm": 0.22655190527439117,
+      "learning_rate": 1.8671960297766752e-05,
+      "loss": 0.2087,
+      "step": 670
+    },
+    {
+      "epoch": 0.337468982630273,
+      "grad_norm": 2.4306397438049316,
+      "learning_rate": 1.865210918114144e-05,
+      "loss": 0.1034,
+      "step": 680
+    },
+    {
+      "epoch": 0.3424317617866005,
+      "grad_norm": 0.11680830270051956,
+      "learning_rate": 1.863225806451613e-05,
+      "loss": 0.0316,
+      "step": 690
+    },
+    {
+      "epoch": 0.34739454094292804,
+      "grad_norm": 0.27029949426651,
+      "learning_rate": 1.861240694789082e-05,
+      "loss": 0.131,
+      "step": 700
+    },
+    {
+      "epoch": 0.3523573200992556,
+      "grad_norm": 0.31934475898742676,
+      "learning_rate": 1.859255583126551e-05,
+      "loss": 0.1961,
+      "step": 710
+    },
+    {
+      "epoch": 0.3573200992555831,
+      "grad_norm": 6.141274929046631,
+      "learning_rate": 1.85727047146402e-05,
+      "loss": 0.2118,
+      "step": 720
+    },
+    {
+      "epoch": 0.36228287841191065,
+      "grad_norm": 0.8305376172065735,
+      "learning_rate": 1.855285359801489e-05,
+      "loss": 0.0207,
+      "step": 730
+    },
+    {
+      "epoch": 0.36724565756823824,
+      "grad_norm": 1.411669135093689,
+      "learning_rate": 1.853300248138958e-05,
+      "loss": 0.0639,
+      "step": 740
+    },
+    {
+      "epoch": 0.37220843672456577,
+      "grad_norm": 2.125790596008301,
+      "learning_rate": 1.851315136476427e-05,
+      "loss": 0.2443,
+      "step": 750
+    },
+    {
+      "epoch": 0.3771712158808933,
+      "grad_norm": 0.9730265736579895,
+      "learning_rate": 1.849330024813896e-05,
+      "loss": 0.0786,
+      "step": 760
+    },
+    {
+      "epoch": 0.38213399503722084,
+      "grad_norm": 3.10215163230896,
+      "learning_rate": 1.847344913151365e-05,
+      "loss": 0.3467,
+      "step": 770
+    },
+    {
+      "epoch": 0.3870967741935484,
+      "grad_norm": 3.7202343940734863,
+      "learning_rate": 1.845359801488834e-05,
+      "loss": 0.2285,
+      "step": 780
+    },
+    {
+      "epoch": 0.3920595533498759,
+      "grad_norm": 0.5115141272544861,
+      "learning_rate": 1.843374689826303e-05,
+      "loss": 0.1705,
+      "step": 790
+    },
+    {
+      "epoch": 0.3970223325062035,
+      "grad_norm": 0.4430786669254303,
+      "learning_rate": 1.8413895781637717e-05,
+      "loss": 0.0631,
+      "step": 800
+    },
+    {
+      "epoch": 0.40198511166253104,
+      "grad_norm": 4.2651143074035645,
+      "learning_rate": 1.8394044665012408e-05,
+      "loss": 0.1615,
+      "step": 810
+    },
+    {
+      "epoch": 0.40694789081885857,
+      "grad_norm": 5.431081295013428,
+      "learning_rate": 1.83741935483871e-05,
+      "loss": 0.0325,
+      "step": 820
+    },
+    {
+      "epoch": 0.4119106699751861,
+      "grad_norm": 1.7246816158294678,
+      "learning_rate": 1.835434243176179e-05,
+      "loss": 0.0818,
+      "step": 830
+    },
+    {
+      "epoch": 0.41687344913151364,
+      "grad_norm": 0.11597002297639847,
+      "learning_rate": 1.8334491315136478e-05,
+      "loss": 0.1661,
+      "step": 840
+    },
+    {
+      "epoch": 0.4218362282878412,
+      "grad_norm": 5.003543853759766,
+      "learning_rate": 1.831464019851117e-05,
+      "loss": 0.0775,
+      "step": 850
+    },
+    {
+      "epoch": 0.4267990074441687,
+      "grad_norm": 6.160369396209717,
+      "learning_rate": 1.8294789081885856e-05,
+      "loss": 0.0777,
+      "step": 860
+    },
+    {
+      "epoch": 0.4317617866004963,
+      "grad_norm": 0.4120528995990753,
+      "learning_rate": 1.8274937965260547e-05,
+      "loss": 0.0515,
+      "step": 870
+    },
+    {
+      "epoch": 0.43672456575682383,
+      "grad_norm": 0.1204703077673912,
+      "learning_rate": 1.8255086848635238e-05,
+      "loss": 0.0356,
+      "step": 880
+    },
+    {
+      "epoch": 0.44168734491315137,
+      "grad_norm": 0.09946464747190475,
+      "learning_rate": 1.8235235732009925e-05,
+      "loss": 0.1525,
+      "step": 890
+    },
+    {
+      "epoch": 0.4466501240694789,
+      "grad_norm": 1.1095144748687744,
+      "learning_rate": 1.8215384615384616e-05,
+      "loss": 0.098,
+      "step": 900
+    },
+    {
+      "epoch": 0.45161290322580644,
+      "grad_norm": 0.15330803394317627,
+      "learning_rate": 1.8195533498759307e-05,
+      "loss": 0.0986,
+      "step": 910
+    },
+    {
+      "epoch": 0.456575682382134,
+      "grad_norm": 0.231792613863945,
+      "learning_rate": 1.8175682382133998e-05,
+      "loss": 0.1464,
+      "step": 920
+    },
+    {
+      "epoch": 0.46153846153846156,
+      "grad_norm": 0.15597473084926605,
+      "learning_rate": 1.8155831265508686e-05,
+      "loss": 0.166,
+      "step": 930
+    },
+    {
+      "epoch": 0.4665012406947891,
+      "grad_norm": 1.0127346515655518,
+      "learning_rate": 1.8135980148883377e-05,
+      "loss": 0.1167,
+      "step": 940
+    },
+    {
+      "epoch": 0.47146401985111663,
+      "grad_norm": 2.508481740951538,
+      "learning_rate": 1.8116129032258064e-05,
+      "loss": 0.0913,
+      "step": 950
+    },
+    {
+      "epoch": 0.47642679900744417,
+      "grad_norm": 0.06334420293569565,
+      "learning_rate": 1.8096277915632755e-05,
+      "loss": 0.0285,
+      "step": 960
+    },
+    {
+      "epoch": 0.4813895781637717,
+      "grad_norm": 0.1302240639925003,
+      "learning_rate": 1.8076426799007446e-05,
+      "loss": 0.123,
+      "step": 970
+    },
+    {
+      "epoch": 0.48635235732009924,
+      "grad_norm": 1.0986523628234863,
+      "learning_rate": 1.8056575682382137e-05,
+      "loss": 0.0978,
+      "step": 980
+    },
+    {
+      "epoch": 0.4913151364764268,
+      "grad_norm": 0.08456585556268692,
+      "learning_rate": 1.8036724565756824e-05,
+      "loss": 0.253,
+      "step": 990
+    },
+    {
+      "epoch": 0.49627791563275436,
+      "grad_norm": 3.1052098274230957,
+      "learning_rate": 1.8016873449131515e-05,
+      "loss": 0.0929,
+      "step": 1000
+    },
+    {
+      "epoch": 0.5012406947890818,
+      "grad_norm": 2.032149314880371,
+      "learning_rate": 1.7997022332506203e-05,
+      "loss": 0.2004,
+      "step": 1010
+    },
+    {
+      "epoch": 0.5062034739454094,
+      "grad_norm": 0.11190329492092133,
+      "learning_rate": 1.7977171215880894e-05,
+      "loss": 0.0892,
+      "step": 1020
+    },
+    {
+      "epoch": 0.511166253101737,
+      "grad_norm": 3.9147121906280518,
+      "learning_rate": 1.7957320099255585e-05,
+      "loss": 0.1345,
+      "step": 1030
+    },
+    {
+      "epoch": 0.5161290322580645,
+      "grad_norm": 0.1115802675485611,
+      "learning_rate": 1.7937468982630276e-05,
+      "loss": 0.0167,
+      "step": 1040
+    },
+    {
+      "epoch": 0.5210918114143921,
+      "grad_norm": 0.09744033217430115,
+      "learning_rate": 1.7917617866004963e-05,
+      "loss": 0.0726,
+      "step": 1050
+    },
+    {
+      "epoch": 0.5260545905707196,
+      "grad_norm": 8.279594421386719,
+      "learning_rate": 1.7897766749379654e-05,
+      "loss": 0.1851,
+      "step": 1060
+    },
+    {
+      "epoch": 0.5310173697270472,
+      "grad_norm": 0.12151824682950974,
+      "learning_rate": 1.7877915632754345e-05,
+      "loss": 0.1491,
+      "step": 1070
+    },
+    {
+      "epoch": 0.5359801488833746,
+      "grad_norm": 0.8291270136833191,
+      "learning_rate": 1.7858064516129036e-05,
+      "loss": 0.0114,
+      "step": 1080
+    },
+    {
+      "epoch": 0.5409429280397022,
+      "grad_norm": 3.0179080963134766,
+      "learning_rate": 1.7838213399503723e-05,
+      "loss": 0.1433,
+      "step": 1090
+    },
+    {
+      "epoch": 0.5459057071960298,
+      "grad_norm": 3.6688222885131836,
+      "learning_rate": 1.7818362282878414e-05,
+      "loss": 0.2806,
+      "step": 1100
+    },
+    {
+      "epoch": 0.5508684863523573,
+      "grad_norm": 0.08318022638559341,
+      "learning_rate": 1.7798511166253102e-05,
+      "loss": 0.0321,
+      "step": 1110
+    },
+    {
+      "epoch": 0.5558312655086849,
+      "grad_norm": 0.16207154095172882,
+      "learning_rate": 1.7778660049627793e-05,
+      "loss": 0.0684,
+      "step": 1120
+    },
+    {
+      "epoch": 0.5607940446650124,
+      "grad_norm": 7.777888774871826,
+      "learning_rate": 1.7758808933002484e-05,
+      "loss": 0.1594,
+      "step": 1130
+    },
+    {
+      "epoch": 0.56575682382134,
+      "grad_norm": 0.07727096229791641,
+      "learning_rate": 1.7738957816377175e-05,
+      "loss": 0.0785,
+      "step": 1140
+    },
+    {
+      "epoch": 0.5707196029776674,
+      "grad_norm": 0.2622874975204468,
+      "learning_rate": 1.7719106699751862e-05,
+      "loss": 0.1995,
+      "step": 1150
+    },
+    {
+      "epoch": 0.575682382133995,
+      "grad_norm": 3.662970542907715,
+      "learning_rate": 1.7699255583126553e-05,
+      "loss": 0.1673,
+      "step": 1160
+    },
+    {
+      "epoch": 0.5806451612903226,
+      "grad_norm": 7.050316333770752,
+      "learning_rate": 1.767940446650124e-05,
+      "loss": 0.2565,
+      "step": 1170
+    },
+    {
+      "epoch": 0.5856079404466501,
+      "grad_norm": 0.09787000715732574,
+      "learning_rate": 1.765955334987593e-05,
+      "loss": 0.1664,
+      "step": 1180
+    },
+    {
+      "epoch": 0.5905707196029777,
+      "grad_norm": 0.08483708649873734,
+      "learning_rate": 1.7639702233250622e-05,
+      "loss": 0.1026,
+      "step": 1190
+    },
+    {
+      "epoch": 0.5955334987593052,
+      "grad_norm": 2.9109203815460205,
+      "learning_rate": 1.761985111662531e-05,
+      "loss": 0.1508,
+      "step": 1200
+    },
+    {
+      "epoch": 0.6004962779156328,
+      "grad_norm": 0.09153315424919128,
+      "learning_rate": 1.76e-05,
+      "loss": 0.0846,
+      "step": 1210
+    },
+    {
+      "epoch": 0.6054590570719603,
+      "grad_norm": 0.5280864834785461,
+      "learning_rate": 1.7580148883374692e-05,
+      "loss": 0.1233,
+      "step": 1220
+    },
+    {
+      "epoch": 0.6104218362282878,
+      "grad_norm": 0.20713864266872406,
+      "learning_rate": 1.7560297766749383e-05,
+      "loss": 0.0837,
+      "step": 1230
+    },
+    {
+      "epoch": 0.6153846153846154,
+      "grad_norm": 0.20489178597927094,
+      "learning_rate": 1.754044665012407e-05,
+      "loss": 0.0938,
+      "step": 1240
+    },
+    {
+      "epoch": 0.6203473945409429,
+      "grad_norm": 0.0775533989071846,
+      "learning_rate": 1.752059553349876e-05,
+      "loss": 0.0832,
+      "step": 1250
+    },
+    {
+      "epoch": 0.6253101736972705,
+      "grad_norm": 0.16079996526241302,
+      "learning_rate": 1.750074441687345e-05,
+      "loss": 0.1211,
+      "step": 1260
+    },
+    {
+      "epoch": 0.630272952853598,
+      "grad_norm": 0.06369619816541672,
+      "learning_rate": 1.748089330024814e-05,
+      "loss": 0.0561,
+      "step": 1270
+    },
+    {
+      "epoch": 0.6352357320099256,
+      "grad_norm": 2.8441195487976074,
+      "learning_rate": 1.746104218362283e-05,
+      "loss": 0.1201,
+      "step": 1280
+    },
+    {
+      "epoch": 0.6401985111662531,
+      "grad_norm": 6.837566375732422,
+      "learning_rate": 1.744119106699752e-05,
+      "loss": 0.1761,
+      "step": 1290
+    },
+    {
+      "epoch": 0.6451612903225806,
+      "grad_norm": 0.06762542575597763,
+      "learning_rate": 1.742133995037221e-05,
+      "loss": 0.1454,
+      "step": 1300
+    },
+    {
+      "epoch": 0.6501240694789082,
+      "grad_norm": 0.10768511891365051,
+      "learning_rate": 1.74014888337469e-05,
+      "loss": 0.1109,
+      "step": 1310
+    },
+    {
+      "epoch": 0.6550868486352357,
+      "grad_norm": 0.14378197491168976,
+      "learning_rate": 1.7381637717121587e-05,
+      "loss": 0.0964,
+      "step": 1320
+    },
+    {
+      "epoch": 0.6600496277915633,
+      "grad_norm": 5.744873523712158,
+      "learning_rate": 1.736178660049628e-05,
+      "loss": 0.1388,
+      "step": 1330
+    },
+    {
+      "epoch": 0.6650124069478908,
+      "grad_norm": 1.2244545221328735,
+      "learning_rate": 1.734193548387097e-05,
+      "loss": 0.0129,
+      "step": 1340
+    },
+    {
+      "epoch": 0.6699751861042184,
+      "grad_norm": 6.384188175201416,
+      "learning_rate": 1.732208436724566e-05,
+      "loss": 0.3897,
+      "step": 1350
+    },
+    {
+      "epoch": 0.674937965260546,
+      "grad_norm": 11.079994201660156,
+      "learning_rate": 1.7302233250620348e-05,
+      "loss": 0.1177,
+      "step": 1360
+    },
+    {
+      "epoch": 0.6799007444168734,
+      "grad_norm": 0.9940871000289917,
+      "learning_rate": 1.728238213399504e-05,
+      "loss": 0.0522,
+      "step": 1370
+    },
+    {
+      "epoch": 0.684863523573201,
+      "grad_norm": 4.725884914398193,
+      "learning_rate": 1.726253101736973e-05,
+      "loss": 0.2414,
+      "step": 1380
+    },
+    {
+      "epoch": 0.6898263027295285,
+      "grad_norm": 8.218222618103027,
+      "learning_rate": 1.724267990074442e-05,
+      "loss": 0.0943,
+      "step": 1390
+    },
+    {
+      "epoch": 0.6947890818858561,
+      "grad_norm": 0.0562286414206028,
+      "learning_rate": 1.7222828784119108e-05,
+      "loss": 0.0122,
+      "step": 1400
+    },
+    {
+      "epoch": 0.6997518610421837,
+      "grad_norm": 0.0633021891117096,
+      "learning_rate": 1.72029776674938e-05,
+      "loss": 0.0148,
+      "step": 1410
+    },
+    {
+      "epoch": 0.7047146401985112,
+      "grad_norm": 0.23889249563217163,
+      "learning_rate": 1.7183126550868486e-05,
+      "loss": 0.0958,
+      "step": 1420
+    },
+    {
+      "epoch": 0.7096774193548387,
+      "grad_norm": 3.1089024543762207,
+      "learning_rate": 1.7163275434243177e-05,
+      "loss": 0.2354,
+      "step": 1430
+    },
+    {
+      "epoch": 0.7146401985111662,
+      "grad_norm": 0.04075319692492485,
+      "learning_rate": 1.7143424317617868e-05,
+      "loss": 0.1398,
+      "step": 1440
+    },
+    {
+      "epoch": 0.7196029776674938,
+      "grad_norm": 0.2210625857114792,
+      "learning_rate": 1.712357320099256e-05,
+      "loss": 0.0855,
+      "step": 1450
+    },
+    {
+      "epoch": 0.7245657568238213,
+      "grad_norm": 6.384485244750977,
+      "learning_rate": 1.7103722084367247e-05,
+      "loss": 0.2229,
+      "step": 1460
+    },
+    {
+      "epoch": 0.7295285359801489,
+      "grad_norm": 0.10448583960533142,
+      "learning_rate": 1.7083870967741938e-05,
+      "loss": 0.0054,
+      "step": 1470
+    },
+    {
+      "epoch": 0.7344913151364765,
+      "grad_norm": 0.06005546450614929,
+      "learning_rate": 1.7064019851116625e-05,
+      "loss": 0.0955,
+      "step": 1480
+    },
+    {
+      "epoch": 0.739454094292804,
+      "grad_norm": 0.09095001220703125,
+      "learning_rate": 1.7044168734491316e-05,
+      "loss": 0.1075,
+      "step": 1490
+    },
+    {
+      "epoch": 0.7444168734491315,
+      "grad_norm": 0.05384785681962967,
+      "learning_rate": 1.7024317617866007e-05,
+      "loss": 0.1336,
+      "step": 1500
+    },
+    {
+      "epoch": 0.749379652605459,
+      "grad_norm": 3.4298574924468994,
+      "learning_rate": 1.7004466501240694e-05,
+      "loss": 0.0192,
+      "step": 1510
+    },
+    {
+      "epoch": 0.7543424317617866,
+      "grad_norm": 0.11529748886823654,
+      "learning_rate": 1.6984615384615385e-05,
+      "loss": 0.1036,
+      "step": 1520
+    },
+    {
+      "epoch": 0.7593052109181141,
+      "grad_norm": 13.056891441345215,
+      "learning_rate": 1.6964764267990076e-05,
+      "loss": 0.0958,
+      "step": 1530
+    },
+    {
+      "epoch": 0.7642679900744417,
+      "grad_norm": 0.05889654532074928,
+      "learning_rate": 1.6944913151364767e-05,
+      "loss": 0.1754,
+      "step": 1540
+    },
+    {
+      "epoch": 0.7692307692307693,
+      "grad_norm": 5.287586688995361,
+      "learning_rate": 1.6925062034739455e-05,
+      "loss": 0.1773,
+      "step": 1550
+    },
+    {
+      "epoch": 0.7741935483870968,
+      "grad_norm": 4.1205878257751465,
+      "learning_rate": 1.6905210918114146e-05,
+      "loss": 0.2493,
+      "step": 1560
+    },
+    {
+      "epoch": 0.7791563275434243,
+      "grad_norm": 0.19854427874088287,
+      "learning_rate": 1.6885359801488833e-05,
+      "loss": 0.1686,
+      "step": 1570
+    },
+    {
+      "epoch": 0.7841191066997518,
+      "grad_norm": 0.05931422859430313,
+      "learning_rate": 1.6865508684863524e-05,
+      "loss": 0.0397,
+      "step": 1580
+    },
+    {
+      "epoch": 0.7890818858560794,
+      "grad_norm": 0.10935617238283157,
+      "learning_rate": 1.6845657568238215e-05,
+      "loss": 0.0285,
+      "step": 1590
+    },
+    {
+      "epoch": 0.794044665012407,
+      "grad_norm": 0.10486113280057907,
+      "learning_rate": 1.6825806451612906e-05,
+      "loss": 0.0781,
+      "step": 1600
+    },
+    {
+      "epoch": 0.7990074441687345,
+      "grad_norm": 0.7170459032058716,
+      "learning_rate": 1.6805955334987593e-05,
+      "loss": 0.0829,
+      "step": 1610
+    },
+    {
+      "epoch": 0.8039702233250621,
+      "grad_norm": 0.3470781445503235,
+      "learning_rate": 1.6786104218362284e-05,
+      "loss": 0.053,
+      "step": 1620
+    },
+    {
+      "epoch": 0.8089330024813896,
+      "grad_norm": 4.012697219848633,
+      "learning_rate": 1.6766253101736972e-05,
+      "loss": 0.1086,
+      "step": 1630
+    },
+    {
+      "epoch": 0.8138957816377171,
+      "grad_norm": 0.302435040473938,
+      "learning_rate": 1.6746401985111663e-05,
+      "loss": 0.0111,
+      "step": 1640
+    },
+    {
+      "epoch": 0.8188585607940446,
+      "grad_norm": 0.06520848721265793,
+      "learning_rate": 1.6726550868486354e-05,
+      "loss": 0.0726,
+      "step": 1650
+    },
+    {
+      "epoch": 0.8238213399503722,
+      "grad_norm": 0.09845948964357376,
+      "learning_rate": 1.6706699751861045e-05,
+      "loss": 0.0094,
+      "step": 1660
+    },
+    {
+      "epoch": 0.8287841191066998,
+      "grad_norm": 0.03756513074040413,
+      "learning_rate": 1.6686848635235732e-05,
+      "loss": 0.23,
+      "step": 1670
+    },
+    {
+      "epoch": 0.8337468982630273,
+      "grad_norm": 0.39116331934928894,
+      "learning_rate": 1.6666997518610423e-05,
+      "loss": 0.034,
+      "step": 1680
+    },
+    {
+      "epoch": 0.8387096774193549,
+      "grad_norm": 0.8433384299278259,
+      "learning_rate": 1.6647146401985114e-05,
+      "loss": 0.0264,
+      "step": 1690
+    },
+    {
+      "epoch": 0.8436724565756824,
+      "grad_norm": 0.05700427293777466,
+      "learning_rate": 1.6627295285359805e-05,
+      "loss": 0.2487,
+      "step": 1700
+    },
+    {
+      "epoch": 0.8486352357320099,
+      "grad_norm": 8.650578498840332,
+      "learning_rate": 1.6607444168734492e-05,
+      "loss": 0.2098,
+      "step": 1710
+    },
+    {
+      "epoch": 0.8535980148883374,
+      "grad_norm": 0.07815321534872055,
+      "learning_rate": 1.6587593052109183e-05,
+      "loss": 0.0113,
+      "step": 1720
+    },
+    {
+      "epoch": 0.858560794044665,
+      "grad_norm": 0.11031467467546463,
+      "learning_rate": 1.656774193548387e-05,
+      "loss": 0.0744,
+      "step": 1730
+    },
+    {
+      "epoch": 0.8635235732009926,
+      "grad_norm": 0.7240772247314453,
+      "learning_rate": 1.6547890818858562e-05,
+      "loss": 0.1165,
+      "step": 1740
+    },
+    {
+      "epoch": 0.8684863523573201,
+      "grad_norm": 0.03234798088669777,
+      "learning_rate": 1.6528039702233253e-05,
+      "loss": 0.0678,
+      "step": 1750
+    },
+    {
+      "epoch": 0.8734491315136477,
+      "grad_norm": 0.024767836555838585,
+      "learning_rate": 1.6508188585607944e-05,
+      "loss": 0.0519,
+      "step": 1760
+    },
+    {
+      "epoch": 0.8784119106699751,
+      "grad_norm": 0.03122510015964508,
+      "learning_rate": 1.648833746898263e-05,
+      "loss": 0.1386,
+      "step": 1770
+    },
+    {
+      "epoch": 0.8833746898263027,
+      "grad_norm": 0.824370265007019,
+      "learning_rate": 1.6468486352357322e-05,
+      "loss": 0.164,
+      "step": 1780
+    },
+    {
+      "epoch": 0.8883374689826302,
+      "grad_norm": 5.419961452484131,
+      "learning_rate": 1.644863523573201e-05,
+      "loss": 0.1668,
+      "step": 1790
+    },
+    {
+      "epoch": 0.8933002481389578,
+      "grad_norm": 0.03851868212223053,
+      "learning_rate": 1.64287841191067e-05,
+      "loss": 0.0614,
+      "step": 1800
+    },
+    {
+      "epoch": 0.8982630272952854,
+      "grad_norm": 0.06279598921537399,
+      "learning_rate": 1.640893300248139e-05,
+      "loss": 0.0031,
+      "step": 1810
+    },
+    {
+      "epoch": 0.9032258064516129,
+      "grad_norm": 0.04613855481147766,
+      "learning_rate": 1.6389081885856082e-05,
+      "loss": 0.1774,
+      "step": 1820
+    },
+    {
+      "epoch": 0.9081885856079405,
+      "grad_norm": 0.6312947869300842,
+      "learning_rate": 1.636923076923077e-05,
+      "loss": 0.0815,
+      "step": 1830
+    },
+    {
+      "epoch": 0.913151364764268,
+      "grad_norm": 0.161106139421463,
+      "learning_rate": 1.634937965260546e-05,
+      "loss": 0.1977,
+      "step": 1840
+    },
+    {
+      "epoch": 0.9181141439205955,
+      "grad_norm": 0.03545048087835312,
+      "learning_rate": 1.6329528535980152e-05,
+      "loss": 0.0353,
+      "step": 1850
+    },
+    {
+      "epoch": 0.9230769230769231,
+      "grad_norm": 0.1279628425836563,
+      "learning_rate": 1.630967741935484e-05,
+      "loss": 0.0503,
+      "step": 1860
+    },
+    {
+      "epoch": 0.9280397022332506,
+      "grad_norm": 8.406203269958496,
+      "learning_rate": 1.628982630272953e-05,
+      "loss": 0.0463,
+      "step": 1870
+    },
+    {
+      "epoch": 0.9330024813895782,
+      "grad_norm": 10.595025062561035,
+      "learning_rate": 1.6269975186104218e-05,
+      "loss": 0.0173,
+      "step": 1880
+    },
+    {
+      "epoch": 0.9379652605459057,
+      "grad_norm": 0.7097483277320862,
+      "learning_rate": 1.625012406947891e-05,
+      "loss": 0.0596,
+      "step": 1890
+    },
+    {
+      "epoch": 0.9429280397022333,
+      "grad_norm": 7.379908561706543,
+      "learning_rate": 1.62302729528536e-05,
+      "loss": 0.2075,
+      "step": 1900
+    },
+    {
+      "epoch": 0.9478908188585607,
+      "grad_norm": 0.1548600196838379,
+      "learning_rate": 1.621042183622829e-05,
+      "loss": 0.2226,
+      "step": 1910
+    },
+    {
+      "epoch": 0.9528535980148883,
+      "grad_norm": 0.026123059913516045,
+      "learning_rate": 1.6190570719602978e-05,
+      "loss": 0.1507,
+      "step": 1920
+    },
+    {
+      "epoch": 0.9578163771712159,
+      "grad_norm": 0.08188609778881073,
+      "learning_rate": 1.617071960297767e-05,
+      "loss": 0.1326,
+      "step": 1930
+    },
+    {
+      "epoch": 0.9627791563275434,
+      "grad_norm": 0.3817611634731293,
+      "learning_rate": 1.6150868486352356e-05,
+      "loss": 0.2298,
+      "step": 1940
+    },
+    {
+      "epoch": 0.967741935483871,
+      "grad_norm": 0.048138681799173355,
+      "learning_rate": 1.6131017369727047e-05,
+      "loss": 0.0676,
+      "step": 1950
+    },
+    {
+      "epoch": 0.9727047146401985,
+      "grad_norm": 0.06823063641786575,
+      "learning_rate": 1.6111166253101738e-05,
+      "loss": 0.0727,
+      "step": 1960
+    },
+    {
+      "epoch": 0.9776674937965261,
+      "grad_norm": 0.031212667003273964,
+      "learning_rate": 1.609131513647643e-05,
+      "loss": 0.0943,
+      "step": 1970
+    },
+    {
+      "epoch": 0.9826302729528535,
+      "grad_norm": 7.244811534881592,
+      "learning_rate": 1.6071464019851117e-05,
+      "loss": 0.1229,
+      "step": 1980
+    },
+    {
+      "epoch": 0.9875930521091811,
+      "grad_norm": 0.06972146779298782,
+      "learning_rate": 1.6051612903225808e-05,
+      "loss": 0.3056,
+      "step": 1990
+    },
+    {
+      "epoch": 0.9925558312655087,
+      "grad_norm": 0.026395201683044434,
+      "learning_rate": 1.60317617866005e-05,
+      "loss": 0.0214,
+      "step": 2000
+    },
+    {
+      "epoch": 0.9975186104218362,
+      "grad_norm": 1.8401563167572021,
+      "learning_rate": 1.601191066997519e-05,
+      "loss": 0.077,
+      "step": 2010
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 10075,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 5,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2178492222720000.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2015/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:859efea2201c1469208153e54e3977a99dc6a369e5e443e74e5277d4b63a6e19
+size 5841

checkpoint-4030/README.md

@@ -0,0 +1,206 @@
+---
+base_model: protectai/deberta-v3-base-prompt-injection-v2
+library_name: peft
+tags:
+- base_model:adapter:protectai/deberta-v3-base-prompt-injection-v2
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

checkpoint-4030/adapter_config.json

@@ -0,0 +1,44 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "protectai/deberta-v3-base-prompt-injection-v2",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "value_proj",
+    "query_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-4030/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b875250ec0b43b2baaf8754ca8840f3ba4642013e2c5eb257eab93f364ab40f
+size 9450704

checkpoint-4030/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd12ebd87804ebc0471156d42893d622de4f0926b10eda654f840e0d7c815cd2
+size 18930123

checkpoint-4030/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c20f1d87bf9bb373a4405a8d181be7fc5327b30fede7bf2424b9019e77b254e
+size 14645

checkpoint-4030/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6797f473fafcecc0847f8098ea716e877002ae9fda5970d9cc61d2dc2e82c0cd
+size 1383

checkpoint-4030/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:efd7fe1ecd6e791af11d5a14668d52a841f68dbdf724efb88221d4268362a6e2
+size 1465

checkpoint-4030/special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-4030/tokenizer_config.json

@@ -0,0 +1,63 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128000": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "split_by_punct": false,
+  "stride": 0,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}

checkpoint-4030/trainer_state.json

@@ -0,0 +1,2855 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.0,
+  "eval_steps": 500,
+  "global_step": 4030,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.004962779156327543,
+      "grad_norm": NaN,
+      "learning_rate": 1.9982133995037223e-05,
+      "loss": 4.0715,
+      "step": 10
+    },
+    {
+      "epoch": 0.009925558312655087,
+      "grad_norm": 7.404272079467773,
+      "learning_rate": 1.996228287841191e-05,
+      "loss": 2.8811,
+      "step": 20
+    },
+    {
+      "epoch": 0.01488833746898263,
+      "grad_norm": 13.565997123718262,
+      "learning_rate": 1.99424317617866e-05,
+      "loss": 3.1689,
+      "step": 30
+    },
+    {
+      "epoch": 0.019851116625310174,
+      "grad_norm": 15.281591415405273,
+      "learning_rate": 1.9922580645161292e-05,
+      "loss": 3.0516,
+      "step": 40
+    },
+    {
+      "epoch": 0.02481389578163772,
+      "grad_norm": 7.509647369384766,
+      "learning_rate": 1.9902729528535983e-05,
+      "loss": 2.9276,
+      "step": 50
+    },
+    {
+      "epoch": 0.02977667493796526,
+      "grad_norm": 7.238119602203369,
+      "learning_rate": 1.988287841191067e-05,
+      "loss": 2.0641,
+      "step": 60
+    },
+    {
+      "epoch": 0.034739454094292806,
+      "grad_norm": 8.467841148376465,
+      "learning_rate": 1.9863027295285362e-05,
+      "loss": 1.8259,
+      "step": 70
+    },
+    {
+      "epoch": 0.03970223325062035,
+      "grad_norm": 7.360463619232178,
+      "learning_rate": 1.984317617866005e-05,
+      "loss": 1.7238,
+      "step": 80
+    },
+    {
+      "epoch": 0.04466501240694789,
+      "grad_norm": 6.133955478668213,
+      "learning_rate": 1.982332506203474e-05,
+      "loss": 1.3318,
+      "step": 90
+    },
+    {
+      "epoch": 0.04962779156327544,
+      "grad_norm": 6.2159295082092285,
+      "learning_rate": 1.980347394540943e-05,
+      "loss": 1.1172,
+      "step": 100
+    },
+    {
+      "epoch": 0.05459057071960298,
+      "grad_norm": 2.4206221103668213,
+      "learning_rate": 1.9783622828784122e-05,
+      "loss": 1.033,
+      "step": 110
+    },
+    {
+      "epoch": 0.05955334987593052,
+      "grad_norm": 5.191859245300293,
+      "learning_rate": 1.976377171215881e-05,
+      "loss": 0.98,
+      "step": 120
+    },
+    {
+      "epoch": 0.06451612903225806,
+      "grad_norm": 2.742246150970459,
+      "learning_rate": 1.97439205955335e-05,
+      "loss": 0.7428,
+      "step": 130
+    },
+    {
+      "epoch": 0.06947890818858561,
+      "grad_norm": 4.614717960357666,
+      "learning_rate": 1.972406947890819e-05,
+      "loss": 0.9041,
+      "step": 140
+    },
+    {
+      "epoch": 0.07444168734491315,
+      "grad_norm": 3.7935266494750977,
+      "learning_rate": 1.9704218362282882e-05,
+      "loss": 0.5919,
+      "step": 150
+    },
+    {
+      "epoch": 0.0794044665012407,
+      "grad_norm": 3.5459160804748535,
+      "learning_rate": 1.968436724565757e-05,
+      "loss": 0.7932,
+      "step": 160
+    },
+    {
+      "epoch": 0.08436724565756824,
+      "grad_norm": 1.8404773473739624,
+      "learning_rate": 1.966451612903226e-05,
+      "loss": 0.7619,
+      "step": 170
+    },
+    {
+      "epoch": 0.08933002481389578,
+      "grad_norm": 2.2088851928710938,
+      "learning_rate": 1.964466501240695e-05,
+      "loss": 0.6204,
+      "step": 180
+    },
+    {
+      "epoch": 0.09429280397022333,
+      "grad_norm": 1.566972017288208,
+      "learning_rate": 1.962481389578164e-05,
+      "loss": 0.7081,
+      "step": 190
+    },
+    {
+      "epoch": 0.09925558312655088,
+      "grad_norm": 1.225921630859375,
+      "learning_rate": 1.960496277915633e-05,
+      "loss": 0.5577,
+      "step": 200
+    },
+    {
+      "epoch": 0.10421836228287841,
+      "grad_norm": 2.0241658687591553,
+      "learning_rate": 1.958511166253102e-05,
+      "loss": 0.5101,
+      "step": 210
+    },
+    {
+      "epoch": 0.10918114143920596,
+      "grad_norm": 2.1363158226013184,
+      "learning_rate": 1.956526054590571e-05,
+      "loss": 0.4952,
+      "step": 220
+    },
+    {
+      "epoch": 0.1141439205955335,
+      "grad_norm": 2.7994675636291504,
+      "learning_rate": 1.9545409429280396e-05,
+      "loss": 0.5103,
+      "step": 230
+    },
+    {
+      "epoch": 0.11910669975186104,
+      "grad_norm": 2.4477338790893555,
+      "learning_rate": 1.9525558312655087e-05,
+      "loss": 0.3801,
+      "step": 240
+    },
+    {
+      "epoch": 0.12406947890818859,
+      "grad_norm": 2.0796492099761963,
+      "learning_rate": 1.9505707196029778e-05,
+      "loss": 0.4258,
+      "step": 250
+    },
+    {
+      "epoch": 0.12903225806451613,
+      "grad_norm": 1.3138341903686523,
+      "learning_rate": 1.948585607940447e-05,
+      "loss": 0.4838,
+      "step": 260
+    },
+    {
+      "epoch": 0.13399503722084366,
+      "grad_norm": 1.5272380113601685,
+      "learning_rate": 1.9466004962779156e-05,
+      "loss": 0.366,
+      "step": 270
+    },
+    {
+      "epoch": 0.13895781637717122,
+      "grad_norm": 3.6165964603424072,
+      "learning_rate": 1.9446153846153847e-05,
+      "loss": 0.4171,
+      "step": 280
+    },
+    {
+      "epoch": 0.14392059553349876,
+      "grad_norm": 2.2593328952789307,
+      "learning_rate": 1.9426302729528538e-05,
+      "loss": 0.3028,
+      "step": 290
+    },
+    {
+      "epoch": 0.1488833746898263,
+      "grad_norm": 1.5910056829452515,
+      "learning_rate": 1.940645161290323e-05,
+      "loss": 0.3047,
+      "step": 300
+    },
+    {
+      "epoch": 0.15384615384615385,
+      "grad_norm": 1.767904281616211,
+      "learning_rate": 1.9386600496277917e-05,
+      "loss": 0.4552,
+      "step": 310
+    },
+    {
+      "epoch": 0.1588089330024814,
+      "grad_norm": 1.7085593938827515,
+      "learning_rate": 1.9366749379652608e-05,
+      "loss": 0.2556,
+      "step": 320
+    },
+    {
+      "epoch": 0.16377171215880892,
+      "grad_norm": 1.355582594871521,
+      "learning_rate": 1.9346898263027295e-05,
+      "loss": 0.4337,
+      "step": 330
+    },
+    {
+      "epoch": 0.1687344913151365,
+      "grad_norm": 0.8261783123016357,
+      "learning_rate": 1.9327047146401986e-05,
+      "loss": 0.3328,
+      "step": 340
+    },
+    {
+      "epoch": 0.17369727047146402,
+      "grad_norm": 0.8605831861495972,
+      "learning_rate": 1.9307196029776677e-05,
+      "loss": 0.2587,
+      "step": 350
+    },
+    {
+      "epoch": 0.17866004962779156,
+      "grad_norm": 2.7887861728668213,
+      "learning_rate": 1.9287344913151368e-05,
+      "loss": 0.3559,
+      "step": 360
+    },
+    {
+      "epoch": 0.18362282878411912,
+      "grad_norm": 1.0830411911010742,
+      "learning_rate": 1.9267493796526055e-05,
+      "loss": 0.1826,
+      "step": 370
+    },
+    {
+      "epoch": 0.18858560794044665,
+      "grad_norm": 3.8424558639526367,
+      "learning_rate": 1.9247642679900746e-05,
+      "loss": 0.2587,
+      "step": 380
+    },
+    {
+      "epoch": 0.1935483870967742,
+      "grad_norm": 3.894951343536377,
+      "learning_rate": 1.9227791563275434e-05,
+      "loss": 0.2034,
+      "step": 390
+    },
+    {
+      "epoch": 0.19851116625310175,
+      "grad_norm": 3.139410972595215,
+      "learning_rate": 1.9207940446650125e-05,
+      "loss": 0.2387,
+      "step": 400
+    },
+    {
+      "epoch": 0.20347394540942929,
+      "grad_norm": 1.7462635040283203,
+      "learning_rate": 1.9188089330024816e-05,
+      "loss": 0.2626,
+      "step": 410
+    },
+    {
+      "epoch": 0.20843672456575682,
+      "grad_norm": 4.002986907958984,
+      "learning_rate": 1.9168238213399507e-05,
+      "loss": 0.1782,
+      "step": 420
+    },
+    {
+      "epoch": 0.21339950372208435,
+      "grad_norm": 2.251654624938965,
+      "learning_rate": 1.9148387096774194e-05,
+      "loss": 0.1621,
+      "step": 430
+    },
+    {
+      "epoch": 0.21836228287841192,
+      "grad_norm": 0.6951057314872742,
+      "learning_rate": 1.9128535980148885e-05,
+      "loss": 0.1937,
+      "step": 440
+    },
+    {
+      "epoch": 0.22332506203473945,
+      "grad_norm": 0.8540646433830261,
+      "learning_rate": 1.9108684863523576e-05,
+      "loss": 0.1913,
+      "step": 450
+    },
+    {
+      "epoch": 0.228287841191067,
+      "grad_norm": 0.9929500222206116,
+      "learning_rate": 1.9088833746898267e-05,
+      "loss": 0.2433,
+      "step": 460
+    },
+    {
+      "epoch": 0.23325062034739455,
+      "grad_norm": 3.8289272785186768,
+      "learning_rate": 1.9068982630272954e-05,
+      "loss": 0.1903,
+      "step": 470
+    },
+    {
+      "epoch": 0.23821339950372208,
+      "grad_norm": 4.537591934204102,
+      "learning_rate": 1.9049131513647645e-05,
+      "loss": 0.2136,
+      "step": 480
+    },
+    {
+      "epoch": 0.24317617866004962,
+      "grad_norm": 0.20700696110725403,
+      "learning_rate": 1.9029280397022333e-05,
+      "loss": 0.1888,
+      "step": 490
+    },
+    {
+      "epoch": 0.24813895781637718,
+      "grad_norm": 0.4332411289215088,
+      "learning_rate": 1.9009429280397024e-05,
+      "loss": 0.2358,
+      "step": 500
+    },
+    {
+      "epoch": 0.2531017369727047,
+      "grad_norm": 1.050528645515442,
+      "learning_rate": 1.8989578163771715e-05,
+      "loss": 0.0832,
+      "step": 510
+    },
+    {
+      "epoch": 0.25806451612903225,
+      "grad_norm": 0.22720620036125183,
+      "learning_rate": 1.8969727047146406e-05,
+      "loss": 0.1272,
+      "step": 520
+    },
+    {
+      "epoch": 0.2630272952853598,
+      "grad_norm": 3.576136350631714,
+      "learning_rate": 1.8949875930521093e-05,
+      "loss": 0.2066,
+      "step": 530
+    },
+    {
+      "epoch": 0.2679900744416873,
+      "grad_norm": 0.33265048265457153,
+      "learning_rate": 1.8930024813895784e-05,
+      "loss": 0.0556,
+      "step": 540
+    },
+    {
+      "epoch": 0.2729528535980149,
+      "grad_norm": 0.6722800731658936,
+      "learning_rate": 1.891017369727047e-05,
+      "loss": 0.0807,
+      "step": 550
+    },
+    {
+      "epoch": 0.27791563275434245,
+      "grad_norm": 1.2628779411315918,
+      "learning_rate": 1.8890322580645163e-05,
+      "loss": 0.24,
+      "step": 560
+    },
+    {
+      "epoch": 0.28287841191067,
+      "grad_norm": 0.27271902561187744,
+      "learning_rate": 1.8870471464019853e-05,
+      "loss": 0.077,
+      "step": 570
+    },
+    {
+      "epoch": 0.2878411910669975,
+      "grad_norm": 3.738210916519165,
+      "learning_rate": 1.885062034739454e-05,
+      "loss": 0.395,
+      "step": 580
+    },
+    {
+      "epoch": 0.29280397022332505,
+      "grad_norm": 0.6131235361099243,
+      "learning_rate": 1.8830769230769232e-05,
+      "loss": 0.1971,
+      "step": 590
+    },
+    {
+      "epoch": 0.2977667493796526,
+      "grad_norm": 0.8646567463874817,
+      "learning_rate": 1.8810918114143923e-05,
+      "loss": 0.0669,
+      "step": 600
+    },
+    {
+      "epoch": 0.3027295285359802,
+      "grad_norm": 4.964555263519287,
+      "learning_rate": 1.8791066997518614e-05,
+      "loss": 0.2324,
+      "step": 610
+    },
+    {
+      "epoch": 0.3076923076923077,
+      "grad_norm": 0.5028713941574097,
+      "learning_rate": 1.87712158808933e-05,
+      "loss": 0.1819,
+      "step": 620
+    },
+    {
+      "epoch": 0.31265508684863524,
+      "grad_norm": 0.31731173396110535,
+      "learning_rate": 1.8751364764267992e-05,
+      "loss": 0.1463,
+      "step": 630
+    },
+    {
+      "epoch": 0.3176178660049628,
+      "grad_norm": 0.23767873644828796,
+      "learning_rate": 1.873151364764268e-05,
+      "loss": 0.0874,
+      "step": 640
+    },
+    {
+      "epoch": 0.3225806451612903,
+      "grad_norm": 0.5757867693901062,
+      "learning_rate": 1.871166253101737e-05,
+      "loss": 0.1267,
+      "step": 650
+    },
+    {
+      "epoch": 0.32754342431761785,
+      "grad_norm": 2.5683581829071045,
+      "learning_rate": 1.869181141439206e-05,
+      "loss": 0.0823,
+      "step": 660
+    },
+    {
+      "epoch": 0.3325062034739454,
+      "grad_norm": 0.22655190527439117,
+      "learning_rate": 1.8671960297766752e-05,
+      "loss": 0.2087,
+      "step": 670
+    },
+    {
+      "epoch": 0.337468982630273,
+      "grad_norm": 2.4306397438049316,
+      "learning_rate": 1.865210918114144e-05,
+      "loss": 0.1034,
+      "step": 680
+    },
+    {
+      "epoch": 0.3424317617866005,
+      "grad_norm": 0.11680830270051956,
+      "learning_rate": 1.863225806451613e-05,
+      "loss": 0.0316,
+      "step": 690
+    },
+    {
+      "epoch": 0.34739454094292804,
+      "grad_norm": 0.27029949426651,
+      "learning_rate": 1.861240694789082e-05,
+      "loss": 0.131,
+      "step": 700
+    },
+    {
+      "epoch": 0.3523573200992556,
+      "grad_norm": 0.31934475898742676,
+      "learning_rate": 1.859255583126551e-05,
+      "loss": 0.1961,
+      "step": 710
+    },
+    {
+      "epoch": 0.3573200992555831,
+      "grad_norm": 6.141274929046631,
+      "learning_rate": 1.85727047146402e-05,
+      "loss": 0.2118,
+      "step": 720
+    },
+    {
+      "epoch": 0.36228287841191065,
+      "grad_norm": 0.8305376172065735,
+      "learning_rate": 1.855285359801489e-05,
+      "loss": 0.0207,
+      "step": 730
+    },
+    {
+      "epoch": 0.36724565756823824,
+      "grad_norm": 1.411669135093689,
+      "learning_rate": 1.853300248138958e-05,
+      "loss": 0.0639,
+      "step": 740
+    },
+    {
+      "epoch": 0.37220843672456577,
+      "grad_norm": 2.125790596008301,
+      "learning_rate": 1.851315136476427e-05,
+      "loss": 0.2443,
+      "step": 750
+    },
+    {
+      "epoch": 0.3771712158808933,
+      "grad_norm": 0.9730265736579895,
+      "learning_rate": 1.849330024813896e-05,
+      "loss": 0.0786,
+      "step": 760
+    },
+    {
+      "epoch": 0.38213399503722084,
+      "grad_norm": 3.10215163230896,
+      "learning_rate": 1.847344913151365e-05,
+      "loss": 0.3467,
+      "step": 770
+    },
+    {
+      "epoch": 0.3870967741935484,
+      "grad_norm": 3.7202343940734863,
+      "learning_rate": 1.845359801488834e-05,
+      "loss": 0.2285,
+      "step": 780
+    },
+    {
+      "epoch": 0.3920595533498759,
+      "grad_norm": 0.5115141272544861,
+      "learning_rate": 1.843374689826303e-05,
+      "loss": 0.1705,
+      "step": 790
+    },
+    {
+      "epoch": 0.3970223325062035,
+      "grad_norm": 0.4430786669254303,
+      "learning_rate": 1.8413895781637717e-05,
+      "loss": 0.0631,
+      "step": 800
+    },
+    {
+      "epoch": 0.40198511166253104,
+      "grad_norm": 4.2651143074035645,
+      "learning_rate": 1.8394044665012408e-05,
+      "loss": 0.1615,
+      "step": 810
+    },
+    {
+      "epoch": 0.40694789081885857,
+      "grad_norm": 5.431081295013428,
+      "learning_rate": 1.83741935483871e-05,
+      "loss": 0.0325,
+      "step": 820
+    },
+    {
+      "epoch": 0.4119106699751861,
+      "grad_norm": 1.7246816158294678,
+      "learning_rate": 1.835434243176179e-05,
+      "loss": 0.0818,
+      "step": 830
+    },
+    {
+      "epoch": 0.41687344913151364,
+      "grad_norm": 0.11597002297639847,
+      "learning_rate": 1.8334491315136478e-05,
+      "loss": 0.1661,
+      "step": 840
+    },
+    {
+      "epoch": 0.4218362282878412,
+      "grad_norm": 5.003543853759766,
+      "learning_rate": 1.831464019851117e-05,
+      "loss": 0.0775,
+      "step": 850
+    },
+    {
+      "epoch": 0.4267990074441687,
+      "grad_norm": 6.160369396209717,
+      "learning_rate": 1.8294789081885856e-05,
+      "loss": 0.0777,
+      "step": 860
+    },
+    {
+      "epoch": 0.4317617866004963,
+      "grad_norm": 0.4120528995990753,
+      "learning_rate": 1.8274937965260547e-05,
+      "loss": 0.0515,
+      "step": 870
+    },
+    {
+      "epoch": 0.43672456575682383,
+      "grad_norm": 0.1204703077673912,
+      "learning_rate": 1.8255086848635238e-05,
+      "loss": 0.0356,
+      "step": 880
+    },
+    {
+      "epoch": 0.44168734491315137,
+      "grad_norm": 0.09946464747190475,
+      "learning_rate": 1.8235235732009925e-05,
+      "loss": 0.1525,
+      "step": 890
+    },
+    {
+      "epoch": 0.4466501240694789,
+      "grad_norm": 1.1095144748687744,
+      "learning_rate": 1.8215384615384616e-05,
+      "loss": 0.098,
+      "step": 900
+    },
+    {
+      "epoch": 0.45161290322580644,
+      "grad_norm": 0.15330803394317627,
+      "learning_rate": 1.8195533498759307e-05,
+      "loss": 0.0986,
+      "step": 910
+    },
+    {
+      "epoch": 0.456575682382134,
+      "grad_norm": 0.231792613863945,
+      "learning_rate": 1.8175682382133998e-05,
+      "loss": 0.1464,
+      "step": 920
+    },
+    {
+      "epoch": 0.46153846153846156,
+      "grad_norm": 0.15597473084926605,
+      "learning_rate": 1.8155831265508686e-05,
+      "loss": 0.166,
+      "step": 930
+    },
+    {
+      "epoch": 0.4665012406947891,
+      "grad_norm": 1.0127346515655518,
+      "learning_rate": 1.8135980148883377e-05,
+      "loss": 0.1167,
+      "step": 940
+    },
+    {
+      "epoch": 0.47146401985111663,
+      "grad_norm": 2.508481740951538,
+      "learning_rate": 1.8116129032258064e-05,
+      "loss": 0.0913,
+      "step": 950
+    },
+    {
+      "epoch": 0.47642679900744417,
+      "grad_norm": 0.06334420293569565,
+      "learning_rate": 1.8096277915632755e-05,
+      "loss": 0.0285,
+      "step": 960
+    },
+    {
+      "epoch": 0.4813895781637717,
+      "grad_norm": 0.1302240639925003,
+      "learning_rate": 1.8076426799007446e-05,
+      "loss": 0.123,
+      "step": 970
+    },
+    {
+      "epoch": 0.48635235732009924,
+      "grad_norm": 1.0986523628234863,
+      "learning_rate": 1.8056575682382137e-05,
+      "loss": 0.0978,
+      "step": 980
+    },
+    {
+      "epoch": 0.4913151364764268,
+      "grad_norm": 0.08456585556268692,
+      "learning_rate": 1.8036724565756824e-05,
+      "loss": 0.253,
+      "step": 990
+    },
+    {
+      "epoch": 0.49627791563275436,
+      "grad_norm": 3.1052098274230957,
+      "learning_rate": 1.8016873449131515e-05,
+      "loss": 0.0929,
+      "step": 1000
+    },
+    {
+      "epoch": 0.5012406947890818,
+      "grad_norm": 2.032149314880371,
+      "learning_rate": 1.7997022332506203e-05,
+      "loss": 0.2004,
+      "step": 1010
+    },
+    {
+      "epoch": 0.5062034739454094,
+      "grad_norm": 0.11190329492092133,
+      "learning_rate": 1.7977171215880894e-05,
+      "loss": 0.0892,
+      "step": 1020
+    },
+    {
+      "epoch": 0.511166253101737,
+      "grad_norm": 3.9147121906280518,
+      "learning_rate": 1.7957320099255585e-05,
+      "loss": 0.1345,
+      "step": 1030
+    },
+    {
+      "epoch": 0.5161290322580645,
+      "grad_norm": 0.1115802675485611,
+      "learning_rate": 1.7937468982630276e-05,
+      "loss": 0.0167,
+      "step": 1040
+    },
+    {
+      "epoch": 0.5210918114143921,
+      "grad_norm": 0.09744033217430115,
+      "learning_rate": 1.7917617866004963e-05,
+      "loss": 0.0726,
+      "step": 1050
+    },
+    {
+      "epoch": 0.5260545905707196,
+      "grad_norm": 8.279594421386719,
+      "learning_rate": 1.7897766749379654e-05,
+      "loss": 0.1851,
+      "step": 1060
+    },
+    {
+      "epoch": 0.5310173697270472,
+      "grad_norm": 0.12151824682950974,
+      "learning_rate": 1.7877915632754345e-05,
+      "loss": 0.1491,
+      "step": 1070
+    },
+    {
+      "epoch": 0.5359801488833746,
+      "grad_norm": 0.8291270136833191,
+      "learning_rate": 1.7858064516129036e-05,
+      "loss": 0.0114,
+      "step": 1080
+    },
+    {
+      "epoch": 0.5409429280397022,
+      "grad_norm": 3.0179080963134766,
+      "learning_rate": 1.7838213399503723e-05,
+      "loss": 0.1433,
+      "step": 1090
+    },
+    {
+      "epoch": 0.5459057071960298,
+      "grad_norm": 3.6688222885131836,
+      "learning_rate": 1.7818362282878414e-05,
+      "loss": 0.2806,
+      "step": 1100
+    },
+    {
+      "epoch": 0.5508684863523573,
+      "grad_norm": 0.08318022638559341,
+      "learning_rate": 1.7798511166253102e-05,
+      "loss": 0.0321,
+      "step": 1110
+    },
+    {
+      "epoch": 0.5558312655086849,
+      "grad_norm": 0.16207154095172882,
+      "learning_rate": 1.7778660049627793e-05,
+      "loss": 0.0684,
+      "step": 1120
+    },
+    {
+      "epoch": 0.5607940446650124,
+      "grad_norm": 7.777888774871826,
+      "learning_rate": 1.7758808933002484e-05,
+      "loss": 0.1594,
+      "step": 1130
+    },
+    {
+      "epoch": 0.56575682382134,
+      "grad_norm": 0.07727096229791641,
+      "learning_rate": 1.7738957816377175e-05,
+      "loss": 0.0785,
+      "step": 1140
+    },
+    {
+      "epoch": 0.5707196029776674,
+      "grad_norm": 0.2622874975204468,
+      "learning_rate": 1.7719106699751862e-05,
+      "loss": 0.1995,
+      "step": 1150
+    },
+    {
+      "epoch": 0.575682382133995,
+      "grad_norm": 3.662970542907715,
+      "learning_rate": 1.7699255583126553e-05,
+      "loss": 0.1673,
+      "step": 1160
+    },
+    {
+      "epoch": 0.5806451612903226,
+      "grad_norm": 7.050316333770752,
+      "learning_rate": 1.767940446650124e-05,
+      "loss": 0.2565,
+      "step": 1170
+    },
+    {
+      "epoch": 0.5856079404466501,
+      "grad_norm": 0.09787000715732574,
+      "learning_rate": 1.765955334987593e-05,
+      "loss": 0.1664,
+      "step": 1180
+    },
+    {
+      "epoch": 0.5905707196029777,
+      "grad_norm": 0.08483708649873734,
+      "learning_rate": 1.7639702233250622e-05,
+      "loss": 0.1026,
+      "step": 1190
+    },
+    {
+      "epoch": 0.5955334987593052,
+      "grad_norm": 2.9109203815460205,
+      "learning_rate": 1.761985111662531e-05,
+      "loss": 0.1508,
+      "step": 1200
+    },
+    {
+      "epoch": 0.6004962779156328,
+      "grad_norm": 0.09153315424919128,
+      "learning_rate": 1.76e-05,
+      "loss": 0.0846,
+      "step": 1210
+    },
+    {
+      "epoch": 0.6054590570719603,
+      "grad_norm": 0.5280864834785461,
+      "learning_rate": 1.7580148883374692e-05,
+      "loss": 0.1233,
+      "step": 1220
+    },
+    {
+      "epoch": 0.6104218362282878,
+      "grad_norm": 0.20713864266872406,
+      "learning_rate": 1.7560297766749383e-05,
+      "loss": 0.0837,
+      "step": 1230
+    },
+    {
+      "epoch": 0.6153846153846154,
+      "grad_norm": 0.20489178597927094,
+      "learning_rate": 1.754044665012407e-05,
+      "loss": 0.0938,
+      "step": 1240
+    },
+    {
+      "epoch": 0.6203473945409429,
+      "grad_norm": 0.0775533989071846,
+      "learning_rate": 1.752059553349876e-05,
+      "loss": 0.0832,
+      "step": 1250
+    },
+    {
+      "epoch": 0.6253101736972705,
+      "grad_norm": 0.16079996526241302,
+      "learning_rate": 1.750074441687345e-05,
+      "loss": 0.1211,
+      "step": 1260
+    },
+    {
+      "epoch": 0.630272952853598,
+      "grad_norm": 0.06369619816541672,
+      "learning_rate": 1.748089330024814e-05,
+      "loss": 0.0561,
+      "step": 1270
+    },
+    {
+      "epoch": 0.6352357320099256,
+      "grad_norm": 2.8441195487976074,
+      "learning_rate": 1.746104218362283e-05,
+      "loss": 0.1201,
+      "step": 1280
+    },
+    {
+      "epoch": 0.6401985111662531,
+      "grad_norm": 6.837566375732422,
+      "learning_rate": 1.744119106699752e-05,
+      "loss": 0.1761,
+      "step": 1290
+    },
+    {
+      "epoch": 0.6451612903225806,
+      "grad_norm": 0.06762542575597763,
+      "learning_rate": 1.742133995037221e-05,
+      "loss": 0.1454,
+      "step": 1300
+    },
+    {
+      "epoch": 0.6501240694789082,
+      "grad_norm": 0.10768511891365051,
+      "learning_rate": 1.74014888337469e-05,
+      "loss": 0.1109,
+      "step": 1310
+    },
+    {
+      "epoch": 0.6550868486352357,
+      "grad_norm": 0.14378197491168976,
+      "learning_rate": 1.7381637717121587e-05,
+      "loss": 0.0964,
+      "step": 1320
+    },
+    {
+      "epoch": 0.6600496277915633,
+      "grad_norm": 5.744873523712158,
+      "learning_rate": 1.736178660049628e-05,
+      "loss": 0.1388,
+      "step": 1330
+    },
+    {
+      "epoch": 0.6650124069478908,
+      "grad_norm": 1.2244545221328735,
+      "learning_rate": 1.734193548387097e-05,
+      "loss": 0.0129,
+      "step": 1340
+    },
+    {
+      "epoch": 0.6699751861042184,
+      "grad_norm": 6.384188175201416,
+      "learning_rate": 1.732208436724566e-05,
+      "loss": 0.3897,
+      "step": 1350
+    },
+    {
+      "epoch": 0.674937965260546,
+      "grad_norm": 11.079994201660156,
+      "learning_rate": 1.7302233250620348e-05,
+      "loss": 0.1177,
+      "step": 1360
+    },
+    {
+      "epoch": 0.6799007444168734,
+      "grad_norm": 0.9940871000289917,
+      "learning_rate": 1.728238213399504e-05,
+      "loss": 0.0522,
+      "step": 1370
+    },
+    {
+      "epoch": 0.684863523573201,
+      "grad_norm": 4.725884914398193,
+      "learning_rate": 1.726253101736973e-05,
+      "loss": 0.2414,
+      "step": 1380
+    },
+    {
+      "epoch": 0.6898263027295285,
+      "grad_norm": 8.218222618103027,
+      "learning_rate": 1.724267990074442e-05,
+      "loss": 0.0943,
+      "step": 1390
+    },
+    {
+      "epoch": 0.6947890818858561,
+      "grad_norm": 0.0562286414206028,
+      "learning_rate": 1.7222828784119108e-05,
+      "loss": 0.0122,
+      "step": 1400
+    },
+    {
+      "epoch": 0.6997518610421837,
+      "grad_norm": 0.0633021891117096,
+      "learning_rate": 1.72029776674938e-05,
+      "loss": 0.0148,
+      "step": 1410
+    },
+    {
+      "epoch": 0.7047146401985112,
+      "grad_norm": 0.23889249563217163,
+      "learning_rate": 1.7183126550868486e-05,
+      "loss": 0.0958,
+      "step": 1420
+    },
+    {
+      "epoch": 0.7096774193548387,
+      "grad_norm": 3.1089024543762207,
+      "learning_rate": 1.7163275434243177e-05,
+      "loss": 0.2354,
+      "step": 1430
+    },
+    {
+      "epoch": 0.7146401985111662,
+      "grad_norm": 0.04075319692492485,
+      "learning_rate": 1.7143424317617868e-05,
+      "loss": 0.1398,
+      "step": 1440
+    },
+    {
+      "epoch": 0.7196029776674938,
+      "grad_norm": 0.2210625857114792,
+      "learning_rate": 1.712357320099256e-05,
+      "loss": 0.0855,
+      "step": 1450
+    },
+    {
+      "epoch": 0.7245657568238213,
+      "grad_norm": 6.384485244750977,
+      "learning_rate": 1.7103722084367247e-05,
+      "loss": 0.2229,
+      "step": 1460
+    },
+    {
+      "epoch": 0.7295285359801489,
+      "grad_norm": 0.10448583960533142,
+      "learning_rate": 1.7083870967741938e-05,
+      "loss": 0.0054,
+      "step": 1470
+    },
+    {
+      "epoch": 0.7344913151364765,
+      "grad_norm": 0.06005546450614929,
+      "learning_rate": 1.7064019851116625e-05,
+      "loss": 0.0955,
+      "step": 1480
+    },
+    {
+      "epoch": 0.739454094292804,
+      "grad_norm": 0.09095001220703125,
+      "learning_rate": 1.7044168734491316e-05,
+      "loss": 0.1075,
+      "step": 1490
+    },
+    {
+      "epoch": 0.7444168734491315,
+      "grad_norm": 0.05384785681962967,
+      "learning_rate": 1.7024317617866007e-05,
+      "loss": 0.1336,
+      "step": 1500
+    },
+    {
+      "epoch": 0.749379652605459,
+      "grad_norm": 3.4298574924468994,
+      "learning_rate": 1.7004466501240694e-05,
+      "loss": 0.0192,
+      "step": 1510
+    },
+    {
+      "epoch": 0.7543424317617866,
+      "grad_norm": 0.11529748886823654,
+      "learning_rate": 1.6984615384615385e-05,
+      "loss": 0.1036,
+      "step": 1520
+    },
+    {
+      "epoch": 0.7593052109181141,
+      "grad_norm": 13.056891441345215,
+      "learning_rate": 1.6964764267990076e-05,
+      "loss": 0.0958,
+      "step": 1530
+    },
+    {
+      "epoch": 0.7642679900744417,
+      "grad_norm": 0.05889654532074928,
+      "learning_rate": 1.6944913151364767e-05,
+      "loss": 0.1754,
+      "step": 1540
+    },
+    {
+      "epoch": 0.7692307692307693,
+      "grad_norm": 5.287586688995361,
+      "learning_rate": 1.6925062034739455e-05,
+      "loss": 0.1773,
+      "step": 1550
+    },
+    {
+      "epoch": 0.7741935483870968,
+      "grad_norm": 4.1205878257751465,
+      "learning_rate": 1.6905210918114146e-05,
+      "loss": 0.2493,
+      "step": 1560
+    },
+    {
+      "epoch": 0.7791563275434243,
+      "grad_norm": 0.19854427874088287,
+      "learning_rate": 1.6885359801488833e-05,
+      "loss": 0.1686,
+      "step": 1570
+    },
+    {
+      "epoch": 0.7841191066997518,
+      "grad_norm": 0.05931422859430313,
+      "learning_rate": 1.6865508684863524e-05,
+      "loss": 0.0397,
+      "step": 1580
+    },
+    {
+      "epoch": 0.7890818858560794,
+      "grad_norm": 0.10935617238283157,
+      "learning_rate": 1.6845657568238215e-05,
+      "loss": 0.0285,
+      "step": 1590
+    },
+    {
+      "epoch": 0.794044665012407,
+      "grad_norm": 0.10486113280057907,
+      "learning_rate": 1.6825806451612906e-05,
+      "loss": 0.0781,
+      "step": 1600
+    },
+    {
+      "epoch": 0.7990074441687345,
+      "grad_norm": 0.7170459032058716,
+      "learning_rate": 1.6805955334987593e-05,
+      "loss": 0.0829,
+      "step": 1610
+    },
+    {
+      "epoch": 0.8039702233250621,
+      "grad_norm": 0.3470781445503235,
+      "learning_rate": 1.6786104218362284e-05,
+      "loss": 0.053,
+      "step": 1620
+    },
+    {
+      "epoch": 0.8089330024813896,
+      "grad_norm": 4.012697219848633,
+      "learning_rate": 1.6766253101736972e-05,
+      "loss": 0.1086,
+      "step": 1630
+    },
+    {
+      "epoch": 0.8138957816377171,
+      "grad_norm": 0.302435040473938,
+      "learning_rate": 1.6746401985111663e-05,
+      "loss": 0.0111,
+      "step": 1640
+    },
+    {
+      "epoch": 0.8188585607940446,
+      "grad_norm": 0.06520848721265793,
+      "learning_rate": 1.6726550868486354e-05,
+      "loss": 0.0726,
+      "step": 1650
+    },
+    {
+      "epoch": 0.8238213399503722,
+      "grad_norm": 0.09845948964357376,
+      "learning_rate": 1.6706699751861045e-05,
+      "loss": 0.0094,
+      "step": 1660
+    },
+    {
+      "epoch": 0.8287841191066998,
+      "grad_norm": 0.03756513074040413,
+      "learning_rate": 1.6686848635235732e-05,
+      "loss": 0.23,
+      "step": 1670
+    },
+    {
+      "epoch": 0.8337468982630273,
+      "grad_norm": 0.39116331934928894,
+      "learning_rate": 1.6666997518610423e-05,
+      "loss": 0.034,
+      "step": 1680
+    },
+    {
+      "epoch": 0.8387096774193549,
+      "grad_norm": 0.8433384299278259,
+      "learning_rate": 1.6647146401985114e-05,
+      "loss": 0.0264,
+      "step": 1690
+    },
+    {
+      "epoch": 0.8436724565756824,
+      "grad_norm": 0.05700427293777466,
+      "learning_rate": 1.6627295285359805e-05,
+      "loss": 0.2487,
+      "step": 1700
+    },
+    {
+      "epoch": 0.8486352357320099,
+      "grad_norm": 8.650578498840332,
+      "learning_rate": 1.6607444168734492e-05,
+      "loss": 0.2098,
+      "step": 1710
+    },
+    {
+      "epoch": 0.8535980148883374,
+      "grad_norm": 0.07815321534872055,
+      "learning_rate": 1.6587593052109183e-05,
+      "loss": 0.0113,
+      "step": 1720
+    },
+    {
+      "epoch": 0.858560794044665,
+      "grad_norm": 0.11031467467546463,
+      "learning_rate": 1.656774193548387e-05,
+      "loss": 0.0744,
+      "step": 1730
+    },
+    {
+      "epoch": 0.8635235732009926,
+      "grad_norm": 0.7240772247314453,
+      "learning_rate": 1.6547890818858562e-05,
+      "loss": 0.1165,
+      "step": 1740
+    },
+    {
+      "epoch": 0.8684863523573201,
+      "grad_norm": 0.03234798088669777,
+      "learning_rate": 1.6528039702233253e-05,
+      "loss": 0.0678,
+      "step": 1750
+    },
+    {
+      "epoch": 0.8734491315136477,
+      "grad_norm": 0.024767836555838585,
+      "learning_rate": 1.6508188585607944e-05,
+      "loss": 0.0519,
+      "step": 1760
+    },
+    {
+      "epoch": 0.8784119106699751,
+      "grad_norm": 0.03122510015964508,
+      "learning_rate": 1.648833746898263e-05,
+      "loss": 0.1386,
+      "step": 1770
+    },
+    {
+      "epoch": 0.8833746898263027,
+      "grad_norm": 0.824370265007019,
+      "learning_rate": 1.6468486352357322e-05,
+      "loss": 0.164,
+      "step": 1780
+    },
+    {
+      "epoch": 0.8883374689826302,
+      "grad_norm": 5.419961452484131,
+      "learning_rate": 1.644863523573201e-05,
+      "loss": 0.1668,
+      "step": 1790
+    },
+    {
+      "epoch": 0.8933002481389578,
+      "grad_norm": 0.03851868212223053,
+      "learning_rate": 1.64287841191067e-05,
+      "loss": 0.0614,
+      "step": 1800
+    },
+    {
+      "epoch": 0.8982630272952854,
+      "grad_norm": 0.06279598921537399,
+      "learning_rate": 1.640893300248139e-05,
+      "loss": 0.0031,
+      "step": 1810
+    },
+    {
+      "epoch": 0.9032258064516129,
+      "grad_norm": 0.04613855481147766,
+      "learning_rate": 1.6389081885856082e-05,
+      "loss": 0.1774,
+      "step": 1820
+    },
+    {
+      "epoch": 0.9081885856079405,
+      "grad_norm": 0.6312947869300842,
+      "learning_rate": 1.636923076923077e-05,
+      "loss": 0.0815,
+      "step": 1830
+    },
+    {
+      "epoch": 0.913151364764268,
+      "grad_norm": 0.161106139421463,
+      "learning_rate": 1.634937965260546e-05,
+      "loss": 0.1977,
+      "step": 1840
+    },
+    {
+      "epoch": 0.9181141439205955,
+      "grad_norm": 0.03545048087835312,
+      "learning_rate": 1.6329528535980152e-05,
+      "loss": 0.0353,
+      "step": 1850
+    },
+    {
+      "epoch": 0.9230769230769231,
+      "grad_norm": 0.1279628425836563,
+      "learning_rate": 1.630967741935484e-05,
+      "loss": 0.0503,
+      "step": 1860
+    },
+    {
+      "epoch": 0.9280397022332506,
+      "grad_norm": 8.406203269958496,
+      "learning_rate": 1.628982630272953e-05,
+      "loss": 0.0463,
+      "step": 1870
+    },
+    {
+      "epoch": 0.9330024813895782,
+      "grad_norm": 10.595025062561035,
+      "learning_rate": 1.6269975186104218e-05,
+      "loss": 0.0173,
+      "step": 1880
+    },
+    {
+      "epoch": 0.9379652605459057,
+      "grad_norm": 0.7097483277320862,
+      "learning_rate": 1.625012406947891e-05,
+      "loss": 0.0596,
+      "step": 1890
+    },
+    {
+      "epoch": 0.9429280397022333,
+      "grad_norm": 7.379908561706543,
+      "learning_rate": 1.62302729528536e-05,
+      "loss": 0.2075,
+      "step": 1900
+    },
+    {
+      "epoch": 0.9478908188585607,
+      "grad_norm": 0.1548600196838379,
+      "learning_rate": 1.621042183622829e-05,
+      "loss": 0.2226,
+      "step": 1910
+    },
+    {
+      "epoch": 0.9528535980148883,
+      "grad_norm": 0.026123059913516045,
+      "learning_rate": 1.6190570719602978e-05,
+      "loss": 0.1507,
+      "step": 1920
+    },
+    {
+      "epoch": 0.9578163771712159,
+      "grad_norm": 0.08188609778881073,
+      "learning_rate": 1.617071960297767e-05,
+      "loss": 0.1326,
+      "step": 1930
+    },
+    {
+      "epoch": 0.9627791563275434,
+      "grad_norm": 0.3817611634731293,
+      "learning_rate": 1.6150868486352356e-05,
+      "loss": 0.2298,
+      "step": 1940
+    },
+    {
+      "epoch": 0.967741935483871,
+      "grad_norm": 0.048138681799173355,
+      "learning_rate": 1.6131017369727047e-05,
+      "loss": 0.0676,
+      "step": 1950
+    },
+    {
+      "epoch": 0.9727047146401985,
+      "grad_norm": 0.06823063641786575,
+      "learning_rate": 1.6111166253101738e-05,
+      "loss": 0.0727,
+      "step": 1960
+    },
+    {
+      "epoch": 0.9776674937965261,
+      "grad_norm": 0.031212667003273964,
+      "learning_rate": 1.609131513647643e-05,
+      "loss": 0.0943,
+      "step": 1970
+    },
+    {
+      "epoch": 0.9826302729528535,
+      "grad_norm": 7.244811534881592,
+      "learning_rate": 1.6071464019851117e-05,
+      "loss": 0.1229,
+      "step": 1980
+    },
+    {
+      "epoch": 0.9875930521091811,
+      "grad_norm": 0.06972146779298782,
+      "learning_rate": 1.6051612903225808e-05,
+      "loss": 0.3056,
+      "step": 1990
+    },
+    {
+      "epoch": 0.9925558312655087,
+      "grad_norm": 0.026395201683044434,
+      "learning_rate": 1.60317617866005e-05,
+      "loss": 0.0214,
+      "step": 2000
+    },
+    {
+      "epoch": 0.9975186104218362,
+      "grad_norm": 1.8401563167572021,
+      "learning_rate": 1.601191066997519e-05,
+      "loss": 0.077,
+      "step": 2010
+    },
+    {
+      "epoch": 1.0024813895781637,
+      "grad_norm": 0.16728228330612183,
+      "learning_rate": 1.5992059553349877e-05,
+      "loss": 0.005,
+      "step": 2020
+    },
+    {
+      "epoch": 1.0074441687344913,
+      "grad_norm": 1.7247531414031982,
+      "learning_rate": 1.5972208436724568e-05,
+      "loss": 0.1244,
+      "step": 2030
+    },
+    {
+      "epoch": 1.0124069478908189,
+      "grad_norm": 0.02384355291724205,
+      "learning_rate": 1.5952357320099255e-05,
+      "loss": 0.1123,
+      "step": 2040
+    },
+    {
+      "epoch": 1.0173697270471465,
+      "grad_norm": 8.84070873260498,
+      "learning_rate": 1.5932506203473946e-05,
+      "loss": 0.067,
+      "step": 2050
+    },
+    {
+      "epoch": 1.022332506203474,
+      "grad_norm": 0.37916916608810425,
+      "learning_rate": 1.5912655086848637e-05,
+      "loss": 0.1113,
+      "step": 2060
+    },
+    {
+      "epoch": 1.0272952853598014,
+      "grad_norm": 0.29101115465164185,
+      "learning_rate": 1.5892803970223328e-05,
+      "loss": 0.0715,
+      "step": 2070
+    },
+    {
+      "epoch": 1.032258064516129,
+      "grad_norm": 3.343764066696167,
+      "learning_rate": 1.5872952853598016e-05,
+      "loss": 0.0682,
+      "step": 2080
+    },
+    {
+      "epoch": 1.0372208436724566,
+      "grad_norm": 5.414521217346191,
+      "learning_rate": 1.5853101736972707e-05,
+      "loss": 0.1941,
+      "step": 2090
+    },
+    {
+      "epoch": 1.0421836228287842,
+      "grad_norm": 4.677765846252441,
+      "learning_rate": 1.5833250620347394e-05,
+      "loss": 0.0441,
+      "step": 2100
+    },
+    {
+      "epoch": 1.0471464019851116,
+      "grad_norm": 0.03460843861103058,
+      "learning_rate": 1.5813399503722085e-05,
+      "loss": 0.0268,
+      "step": 2110
+    },
+    {
+      "epoch": 1.0521091811414391,
+      "grad_norm": 1.8757960796356201,
+      "learning_rate": 1.5793548387096776e-05,
+      "loss": 0.0152,
+      "step": 2120
+    },
+    {
+      "epoch": 1.0570719602977667,
+      "grad_norm": 0.0749705508351326,
+      "learning_rate": 1.5773697270471467e-05,
+      "loss": 0.2066,
+      "step": 2130
+    },
+    {
+      "epoch": 1.0620347394540943,
+      "grad_norm": 0.05351117253303528,
+      "learning_rate": 1.5753846153846154e-05,
+      "loss": 0.0652,
+      "step": 2140
+    },
+    {
+      "epoch": 1.066997518610422,
+      "grad_norm": 0.2384226769208908,
+      "learning_rate": 1.5733995037220845e-05,
+      "loss": 0.0989,
+      "step": 2150
+    },
+    {
+      "epoch": 1.0719602977667493,
+      "grad_norm": 10.485825538635254,
+      "learning_rate": 1.5714143920595536e-05,
+      "loss": 0.1214,
+      "step": 2160
+    },
+    {
+      "epoch": 1.0769230769230769,
+      "grad_norm": 0.02599877305328846,
+      "learning_rate": 1.5694292803970224e-05,
+      "loss": 0.1264,
+      "step": 2170
+    },
+    {
+      "epoch": 1.0818858560794045,
+      "grad_norm": 4.427301406860352,
+      "learning_rate": 1.5674441687344915e-05,
+      "loss": 0.0143,
+      "step": 2180
+    },
+    {
+      "epoch": 1.086848635235732,
+      "grad_norm": 0.05011460930109024,
+      "learning_rate": 1.5654590570719602e-05,
+      "loss": 0.0444,
+      "step": 2190
+    },
+    {
+      "epoch": 1.0918114143920596,
+      "grad_norm": 0.08383812755346298,
+      "learning_rate": 1.5634739454094293e-05,
+      "loss": 0.1946,
+      "step": 2200
+    },
+    {
+      "epoch": 1.096774193548387,
+      "grad_norm": 0.2225169986486435,
+      "learning_rate": 1.5614888337468984e-05,
+      "loss": 0.0471,
+      "step": 2210
+    },
+    {
+      "epoch": 1.1017369727047146,
+      "grad_norm": 0.09526325762271881,
+      "learning_rate": 1.5595037220843675e-05,
+      "loss": 0.162,
+      "step": 2220
+    },
+    {
+      "epoch": 1.1066997518610422,
+      "grad_norm": 0.0817873403429985,
+      "learning_rate": 1.5575186104218363e-05,
+      "loss": 0.0623,
+      "step": 2230
+    },
+    {
+      "epoch": 1.1116625310173698,
+      "grad_norm": 0.02879715897142887,
+      "learning_rate": 1.5555334987593053e-05,
+      "loss": 0.1162,
+      "step": 2240
+    },
+    {
+      "epoch": 1.1166253101736974,
+      "grad_norm": 0.04488246515393257,
+      "learning_rate": 1.553548387096774e-05,
+      "loss": 0.0868,
+      "step": 2250
+    },
+    {
+      "epoch": 1.1215880893300247,
+      "grad_norm": 0.0384470596909523,
+      "learning_rate": 1.5515632754342432e-05,
+      "loss": 0.0585,
+      "step": 2260
+    },
+    {
+      "epoch": 1.1265508684863523,
+      "grad_norm": 4.3215813636779785,
+      "learning_rate": 1.5495781637717123e-05,
+      "loss": 0.0381,
+      "step": 2270
+    },
+    {
+      "epoch": 1.13151364764268,
+      "grad_norm": 2.8902511596679688,
+      "learning_rate": 1.5475930521091814e-05,
+      "loss": 0.0082,
+      "step": 2280
+    },
+    {
+      "epoch": 1.1364764267990075,
+      "grad_norm": 4.735926628112793,
+      "learning_rate": 1.54560794044665e-05,
+      "loss": 0.0722,
+      "step": 2290
+    },
+    {
+      "epoch": 1.141439205955335,
+      "grad_norm": 0.10370062291622162,
+      "learning_rate": 1.5436228287841192e-05,
+      "loss": 0.1362,
+      "step": 2300
+    },
+    {
+      "epoch": 1.1464019851116625,
+      "grad_norm": 0.05682974308729172,
+      "learning_rate": 1.5416377171215883e-05,
+      "loss": 0.0302,
+      "step": 2310
+    },
+    {
+      "epoch": 1.15136476426799,
+      "grad_norm": 0.19948908686637878,
+      "learning_rate": 1.5396526054590574e-05,
+      "loss": 0.0604,
+      "step": 2320
+    },
+    {
+      "epoch": 1.1563275434243176,
+      "grad_norm": 0.09828030318021774,
+      "learning_rate": 1.537667493796526e-05,
+      "loss": 0.1613,
+      "step": 2330
+    },
+    {
+      "epoch": 1.1612903225806452,
+      "grad_norm": 0.11819926649332047,
+      "learning_rate": 1.5356823821339952e-05,
+      "loss": 0.0951,
+      "step": 2340
+    },
+    {
+      "epoch": 1.1662531017369726,
+      "grad_norm": 11.936241149902344,
+      "learning_rate": 1.533697270471464e-05,
+      "loss": 0.026,
+      "step": 2350
+    },
+    {
+      "epoch": 1.1712158808933002,
+      "grad_norm": 0.03054685890674591,
+      "learning_rate": 1.531712158808933e-05,
+      "loss": 0.0211,
+      "step": 2360
+    },
+    {
+      "epoch": 1.1761786600496278,
+      "grad_norm": 0.05654008686542511,
+      "learning_rate": 1.5297270471464022e-05,
+      "loss": 0.1283,
+      "step": 2370
+    },
+    {
+      "epoch": 1.1811414392059554,
+      "grad_norm": 10.214383125305176,
+      "learning_rate": 1.5277419354838713e-05,
+      "loss": 0.1008,
+      "step": 2380
+    },
+    {
+      "epoch": 1.186104218362283,
+      "grad_norm": 0.0567215271294117,
+      "learning_rate": 1.52575682382134e-05,
+      "loss": 0.019,
+      "step": 2390
+    },
+    {
+      "epoch": 1.1910669975186103,
+      "grad_norm": 5.2418036460876465,
+      "learning_rate": 1.5237717121588091e-05,
+      "loss": 0.1212,
+      "step": 2400
+    },
+    {
+      "epoch": 1.196029776674938,
+      "grad_norm": 1.5446528196334839,
+      "learning_rate": 1.521786600496278e-05,
+      "loss": 0.0633,
+      "step": 2410
+    },
+    {
+      "epoch": 1.2009925558312655,
+      "grad_norm": 9.704379081726074,
+      "learning_rate": 1.5198014888337471e-05,
+      "loss": 0.0984,
+      "step": 2420
+    },
+    {
+      "epoch": 1.205955334987593,
+      "grad_norm": 0.023028984665870667,
+      "learning_rate": 1.517816377171216e-05,
+      "loss": 0.0584,
+      "step": 2430
+    },
+    {
+      "epoch": 1.2109181141439205,
+      "grad_norm": 0.18110217154026031,
+      "learning_rate": 1.5158312655086851e-05,
+      "loss": 0.1034,
+      "step": 2440
+    },
+    {
+      "epoch": 1.215880893300248,
+      "grad_norm": 0.10866773873567581,
+      "learning_rate": 1.5138461538461539e-05,
+      "loss": 0.0034,
+      "step": 2450
+    },
+    {
+      "epoch": 1.2208436724565757,
+      "grad_norm": 6.370197296142578,
+      "learning_rate": 1.511861042183623e-05,
+      "loss": 0.015,
+      "step": 2460
+    },
+    {
+      "epoch": 1.2258064516129032,
+      "grad_norm": 0.08329326659440994,
+      "learning_rate": 1.5098759305210919e-05,
+      "loss": 0.0499,
+      "step": 2470
+    },
+    {
+      "epoch": 1.2307692307692308,
+      "grad_norm": 3.680455207824707,
+      "learning_rate": 1.5078908188585608e-05,
+      "loss": 0.1308,
+      "step": 2480
+    },
+    {
+      "epoch": 1.2357320099255582,
+      "grad_norm": 0.029821470379829407,
+      "learning_rate": 1.50590570719603e-05,
+      "loss": 0.0368,
+      "step": 2490
+    },
+    {
+      "epoch": 1.2406947890818858,
+      "grad_norm": 0.022040782496333122,
+      "learning_rate": 1.5039205955334988e-05,
+      "loss": 0.0303,
+      "step": 2500
+    },
+    {
+      "epoch": 1.2456575682382134,
+      "grad_norm": 0.07885632663965225,
+      "learning_rate": 1.501935483870968e-05,
+      "loss": 0.1438,
+      "step": 2510
+    },
+    {
+      "epoch": 1.250620347394541,
+      "grad_norm": 0.03130103647708893,
+      "learning_rate": 1.4999503722084367e-05,
+      "loss": 0.1518,
+      "step": 2520
+    },
+    {
+      "epoch": 1.2555831265508686,
+      "grad_norm": 0.05089149251580238,
+      "learning_rate": 1.4979652605459058e-05,
+      "loss": 0.0356,
+      "step": 2530
+    },
+    {
+      "epoch": 1.260545905707196,
+      "grad_norm": 0.050943199545145035,
+      "learning_rate": 1.4959801488833747e-05,
+      "loss": 0.0363,
+      "step": 2540
+    },
+    {
+      "epoch": 1.2655086848635235,
+      "grad_norm": 0.03484184294939041,
+      "learning_rate": 1.4939950372208438e-05,
+      "loss": 0.0081,
+      "step": 2550
+    },
+    {
+      "epoch": 1.2704714640198511,
+      "grad_norm": 0.06282445788383484,
+      "learning_rate": 1.4920099255583127e-05,
+      "loss": 0.0476,
+      "step": 2560
+    },
+    {
+      "epoch": 1.2754342431761787,
+      "grad_norm": 3.406345844268799,
+      "learning_rate": 1.4900248138957818e-05,
+      "loss": 0.0106,
+      "step": 2570
+    },
+    {
+      "epoch": 1.2803970223325063,
+      "grad_norm": 0.05051049962639809,
+      "learning_rate": 1.4880397022332507e-05,
+      "loss": 0.2119,
+      "step": 2580
+    },
+    {
+      "epoch": 1.2853598014888337,
+      "grad_norm": 0.05261968821287155,
+      "learning_rate": 1.4860545905707198e-05,
+      "loss": 0.0109,
+      "step": 2590
+    },
+    {
+      "epoch": 1.2903225806451613,
+      "grad_norm": 1.54677414894104,
+      "learning_rate": 1.4840694789081886e-05,
+      "loss": 0.0026,
+      "step": 2600
+    },
+    {
+      "epoch": 1.2952853598014888,
+      "grad_norm": 0.010906929150223732,
+      "learning_rate": 1.4820843672456577e-05,
+      "loss": 0.0889,
+      "step": 2610
+    },
+    {
+      "epoch": 1.3002481389578164,
+      "grad_norm": 0.014594710431993008,
+      "learning_rate": 1.4800992555831266e-05,
+      "loss": 0.1475,
+      "step": 2620
+    },
+    {
+      "epoch": 1.305210918114144,
+      "grad_norm": 0.02966361679136753,
+      "learning_rate": 1.4781141439205957e-05,
+      "loss": 0.0145,
+      "step": 2630
+    },
+    {
+      "epoch": 1.3101736972704714,
+      "grad_norm": 0.02814514935016632,
+      "learning_rate": 1.4761290322580646e-05,
+      "loss": 0.1239,
+      "step": 2640
+    },
+    {
+      "epoch": 1.315136476426799,
+      "grad_norm": 0.014956213533878326,
+      "learning_rate": 1.4741439205955337e-05,
+      "loss": 0.0637,
+      "step": 2650
+    },
+    {
+      "epoch": 1.3200992555831266,
+      "grad_norm": 0.01340537704527378,
+      "learning_rate": 1.4721588089330026e-05,
+      "loss": 0.09,
+      "step": 2660
+    },
+    {
+      "epoch": 1.3250620347394542,
+      "grad_norm": 0.031831275671720505,
+      "learning_rate": 1.4701736972704717e-05,
+      "loss": 0.0536,
+      "step": 2670
+    },
+    {
+      "epoch": 1.3300248138957818,
+      "grad_norm": 0.035812411457300186,
+      "learning_rate": 1.4681885856079405e-05,
+      "loss": 0.1342,
+      "step": 2680
+    },
+    {
+      "epoch": 1.3349875930521091,
+      "grad_norm": 0.18592360615730286,
+      "learning_rate": 1.4662034739454096e-05,
+      "loss": 0.0345,
+      "step": 2690
+    },
+    {
+      "epoch": 1.3399503722084367,
+      "grad_norm": 0.07656742632389069,
+      "learning_rate": 1.4642183622828785e-05,
+      "loss": 0.0741,
+      "step": 2700
+    },
+    {
+      "epoch": 1.3449131513647643,
+      "grad_norm": 0.048842281103134155,
+      "learning_rate": 1.4622332506203476e-05,
+      "loss": 0.0996,
+      "step": 2710
+    },
+    {
+      "epoch": 1.349875930521092,
+      "grad_norm": 0.04869871214032173,
+      "learning_rate": 1.4602481389578165e-05,
+      "loss": 0.0219,
+      "step": 2720
+    },
+    {
+      "epoch": 1.3548387096774195,
+      "grad_norm": 0.14760206639766693,
+      "learning_rate": 1.4582630272952856e-05,
+      "loss": 0.1133,
+      "step": 2730
+    },
+    {
+      "epoch": 1.3598014888337469,
+      "grad_norm": 8.338112831115723,
+      "learning_rate": 1.4562779156327545e-05,
+      "loss": 0.0339,
+      "step": 2740
+    },
+    {
+      "epoch": 1.3647642679900744,
+      "grad_norm": 3.8133182525634766,
+      "learning_rate": 1.4542928039702236e-05,
+      "loss": 0.1043,
+      "step": 2750
+    },
+    {
+      "epoch": 1.369727047146402,
+      "grad_norm": 0.024321183562278748,
+      "learning_rate": 1.4523076923076923e-05,
+      "loss": 0.0852,
+      "step": 2760
+    },
+    {
+      "epoch": 1.3746898263027294,
+      "grad_norm": 14.953070640563965,
+      "learning_rate": 1.4503225806451614e-05,
+      "loss": 0.1715,
+      "step": 2770
+    },
+    {
+      "epoch": 1.379652605459057,
+      "grad_norm": 0.029152385890483856,
+      "learning_rate": 1.4483374689826304e-05,
+      "loss": 0.2272,
+      "step": 2780
+    },
+    {
+      "epoch": 1.3846153846153846,
+      "grad_norm": 0.01473495364189148,
+      "learning_rate": 1.4463523573200993e-05,
+      "loss": 0.0318,
+      "step": 2790
+    },
+    {
+      "epoch": 1.3895781637717122,
+      "grad_norm": 0.009415539912879467,
+      "learning_rate": 1.4443672456575684e-05,
+      "loss": 0.017,
+      "step": 2800
+    },
+    {
+      "epoch": 1.3945409429280398,
+      "grad_norm": 0.01470163743942976,
+      "learning_rate": 1.4423821339950373e-05,
+      "loss": 0.0066,
+      "step": 2810
+    },
+    {
+      "epoch": 1.3995037220843671,
+      "grad_norm": 0.09982536733150482,
+      "learning_rate": 1.4403970223325064e-05,
+      "loss": 0.0051,
+      "step": 2820
+    },
+    {
+      "epoch": 1.4044665012406947,
+      "grad_norm": 0.03142406791448593,
+      "learning_rate": 1.4384119106699751e-05,
+      "loss": 0.0129,
+      "step": 2830
+    },
+    {
+      "epoch": 1.4094292803970223,
+      "grad_norm": 0.34097033739089966,
+      "learning_rate": 1.4364267990074442e-05,
+      "loss": 0.0786,
+      "step": 2840
+    },
+    {
+      "epoch": 1.41439205955335,
+      "grad_norm": 5.292384624481201,
+      "learning_rate": 1.4344416873449132e-05,
+      "loss": 0.0771,
+      "step": 2850
+    },
+    {
+      "epoch": 1.4193548387096775,
+      "grad_norm": 0.014468374662101269,
+      "learning_rate": 1.4324565756823822e-05,
+      "loss": 0.2335,
+      "step": 2860
+    },
+    {
+      "epoch": 1.4243176178660049,
+      "grad_norm": 4.065907001495361,
+      "learning_rate": 1.4304714640198512e-05,
+      "loss": 0.0379,
+      "step": 2870
+    },
+    {
+      "epoch": 1.4292803970223324,
+      "grad_norm": 0.17883175611495972,
+      "learning_rate": 1.4284863523573203e-05,
+      "loss": 0.1032,
+      "step": 2880
+    },
+    {
+      "epoch": 1.43424317617866,
+      "grad_norm": 0.16030851006507874,
+      "learning_rate": 1.4265012406947892e-05,
+      "loss": 0.0889,
+      "step": 2890
+    },
+    {
+      "epoch": 1.4392059553349876,
+      "grad_norm": 0.027890929952263832,
+      "learning_rate": 1.4245161290322583e-05,
+      "loss": 0.0775,
+      "step": 2900
+    },
+    {
+      "epoch": 1.4441687344913152,
+      "grad_norm": 0.018904339522123337,
+      "learning_rate": 1.422531017369727e-05,
+      "loss": 0.0695,
+      "step": 2910
+    },
+    {
+      "epoch": 1.4491315136476426,
+      "grad_norm": 0.018008772283792496,
+      "learning_rate": 1.4205459057071961e-05,
+      "loss": 0.0132,
+      "step": 2920
+    },
+    {
+      "epoch": 1.4540942928039702,
+      "grad_norm": 0.06981614977121353,
+      "learning_rate": 1.418560794044665e-05,
+      "loss": 0.0018,
+      "step": 2930
+    },
+    {
+      "epoch": 1.4590570719602978,
+      "grad_norm": 0.014432519674301147,
+      "learning_rate": 1.4165756823821341e-05,
+      "loss": 0.1379,
+      "step": 2940
+    },
+    {
+      "epoch": 1.4640198511166254,
+      "grad_norm": 0.09437421709299088,
+      "learning_rate": 1.414590570719603e-05,
+      "loss": 0.016,
+      "step": 2950
+    },
+    {
+      "epoch": 1.468982630272953,
+      "grad_norm": 7.359278678894043,
+      "learning_rate": 1.4126054590570721e-05,
+      "loss": 0.1921,
+      "step": 2960
+    },
+    {
+      "epoch": 1.4739454094292803,
+      "grad_norm": 0.030443871393799782,
+      "learning_rate": 1.410620347394541e-05,
+      "loss": 0.0919,
+      "step": 2970
+    },
+    {
+      "epoch": 1.478908188585608,
+      "grad_norm": 0.020756110548973083,
+      "learning_rate": 1.4086352357320102e-05,
+      "loss": 0.1741,
+      "step": 2980
+    },
+    {
+      "epoch": 1.4838709677419355,
+      "grad_norm": 10.432821273803711,
+      "learning_rate": 1.4066501240694789e-05,
+      "loss": 0.1364,
+      "step": 2990
+    },
+    {
+      "epoch": 1.488833746898263,
+      "grad_norm": 2.563176393508911,
+      "learning_rate": 1.404665012406948e-05,
+      "loss": 0.1259,
+      "step": 3000
+    },
+    {
+      "epoch": 1.4937965260545907,
+      "grad_norm": 0.11504728347063065,
+      "learning_rate": 1.402679900744417e-05,
+      "loss": 0.0028,
+      "step": 3010
+    },
+    {
+      "epoch": 1.498759305210918,
+      "grad_norm": 0.07825014740228653,
+      "learning_rate": 1.400694789081886e-05,
+      "loss": 0.0029,
+      "step": 3020
+    },
+    {
+      "epoch": 1.5037220843672456,
+      "grad_norm": 0.05102028325200081,
+      "learning_rate": 1.398709677419355e-05,
+      "loss": 0.1108,
+      "step": 3030
+    },
+    {
+      "epoch": 1.5086848635235732,
+      "grad_norm": 0.10214142501354218,
+      "learning_rate": 1.396724565756824e-05,
+      "loss": 0.0138,
+      "step": 3040
+    },
+    {
+      "epoch": 1.5136476426799006,
+      "grad_norm": 0.13721299171447754,
+      "learning_rate": 1.394739454094293e-05,
+      "loss": 0.0175,
+      "step": 3050
+    },
+    {
+      "epoch": 1.5186104218362284,
+      "grad_norm": 0.024655604735016823,
+      "learning_rate": 1.392754342431762e-05,
+      "loss": 0.0084,
+      "step": 3060
+    },
+    {
+      "epoch": 1.5235732009925558,
+      "grad_norm": 0.0303645096719265,
+      "learning_rate": 1.3907692307692308e-05,
+      "loss": 0.022,
+      "step": 3070
+    },
+    {
+      "epoch": 1.5285359801488834,
+      "grad_norm": 0.5544743537902832,
+      "learning_rate": 1.3887841191066999e-05,
+      "loss": 0.0382,
+      "step": 3080
+    },
+    {
+      "epoch": 1.533498759305211,
+      "grad_norm": 0.09961941838264465,
+      "learning_rate": 1.3867990074441688e-05,
+      "loss": 0.0049,
+      "step": 3090
+    },
+    {
+      "epoch": 1.5384615384615383,
+      "grad_norm": 0.05996626988053322,
+      "learning_rate": 1.3848138957816379e-05,
+      "loss": 0.0591,
+      "step": 3100
+    },
+    {
+      "epoch": 1.5434243176178661,
+      "grad_norm": 0.01039404422044754,
+      "learning_rate": 1.3828287841191068e-05,
+      "loss": 0.0516,
+      "step": 3110
+    },
+    {
+      "epoch": 1.5483870967741935,
+      "grad_norm": 0.21032479405403137,
+      "learning_rate": 1.3808436724565757e-05,
+      "loss": 0.12,
+      "step": 3120
+    },
+    {
+      "epoch": 1.553349875930521,
+      "grad_norm": 0.010956893675029278,
+      "learning_rate": 1.3788585607940448e-05,
+      "loss": 0.0044,
+      "step": 3130
+    },
+    {
+      "epoch": 1.5583126550868487,
+      "grad_norm": 8.013826370239258,
+      "learning_rate": 1.3768734491315136e-05,
+      "loss": 0.0719,
+      "step": 3140
+    },
+    {
+      "epoch": 1.563275434243176,
+      "grad_norm": 6.619443893432617,
+      "learning_rate": 1.3748883374689827e-05,
+      "loss": 0.0613,
+      "step": 3150
+    },
+    {
+      "epoch": 1.5682382133995039,
+      "grad_norm": 0.01725432462990284,
+      "learning_rate": 1.3729032258064516e-05,
+      "loss": 0.0175,
+      "step": 3160
+    },
+    {
+      "epoch": 1.5732009925558312,
+      "grad_norm": 0.05100134387612343,
+      "learning_rate": 1.3709181141439207e-05,
+      "loss": 0.0024,
+      "step": 3170
+    },
+    {
+      "epoch": 1.5781637717121588,
+      "grad_norm": 4.458766937255859,
+      "learning_rate": 1.3689330024813896e-05,
+      "loss": 0.2125,
+      "step": 3180
+    },
+    {
+      "epoch": 1.5831265508684864,
+      "grad_norm": 0.03693610802292824,
+      "learning_rate": 1.3669478908188587e-05,
+      "loss": 0.0088,
+      "step": 3190
+    },
+    {
+      "epoch": 1.5880893300248138,
+      "grad_norm": 0.0044688996858894825,
+      "learning_rate": 1.3649627791563276e-05,
+      "loss": 0.0023,
+      "step": 3200
+    },
+    {
+      "epoch": 1.5930521091811416,
+      "grad_norm": 0.5581690073013306,
+      "learning_rate": 1.3629776674937967e-05,
+      "loss": 0.0911,
+      "step": 3210
+    },
+    {
+      "epoch": 1.598014888337469,
+      "grad_norm": 0.07121065258979797,
+      "learning_rate": 1.3609925558312655e-05,
+      "loss": 0.1167,
+      "step": 3220
+    },
+    {
+      "epoch": 1.6029776674937966,
+      "grad_norm": 0.01797034777700901,
+      "learning_rate": 1.3590074441687346e-05,
+      "loss": 0.2361,
+      "step": 3230
+    },
+    {
+      "epoch": 1.6079404466501241,
+      "grad_norm": 0.03692857176065445,
+      "learning_rate": 1.3570223325062035e-05,
+      "loss": 0.0056,
+      "step": 3240
+    },
+    {
+      "epoch": 1.6129032258064515,
+      "grad_norm": 0.07059811800718307,
+      "learning_rate": 1.3550372208436726e-05,
+      "loss": 0.0783,
+      "step": 3250
+    },
+    {
+      "epoch": 1.617866004962779,
+      "grad_norm": 0.02054552175104618,
+      "learning_rate": 1.3530521091811415e-05,
+      "loss": 0.0771,
+      "step": 3260
+    },
+    {
+      "epoch": 1.6228287841191067,
+      "grad_norm": 0.4568418264389038,
+      "learning_rate": 1.3510669975186106e-05,
+      "loss": 0.0927,
+      "step": 3270
+    },
+    {
+      "epoch": 1.6277915632754343,
+      "grad_norm": 0.021787209436297417,
+      "learning_rate": 1.3490818858560795e-05,
+      "loss": 0.0586,
+      "step": 3280
+    },
+    {
+      "epoch": 1.6327543424317619,
+      "grad_norm": 0.043457385152578354,
+      "learning_rate": 1.3470967741935486e-05,
+      "loss": 0.0994,
+      "step": 3290
+    },
+    {
+      "epoch": 1.6377171215880892,
+      "grad_norm": 1.623487114906311,
+      "learning_rate": 1.3451116625310174e-05,
+      "loss": 0.126,
+      "step": 3300
+    },
+    {
+      "epoch": 1.6426799007444168,
+      "grad_norm": 2.6221861839294434,
+      "learning_rate": 1.3431265508684865e-05,
+      "loss": 0.1692,
+      "step": 3310
+    },
+    {
+      "epoch": 1.6476426799007444,
+      "grad_norm": 0.015129676088690758,
+      "learning_rate": 1.3411414392059554e-05,
+      "loss": 0.0484,
+      "step": 3320
+    },
+    {
+      "epoch": 1.652605459057072,
+      "grad_norm": 0.0224574226886034,
+      "learning_rate": 1.3391563275434245e-05,
+      "loss": 0.1117,
+      "step": 3330
+    },
+    {
+      "epoch": 1.6575682382133996,
+      "grad_norm": 0.01793314330279827,
+      "learning_rate": 1.3371712158808934e-05,
+      "loss": 0.0027,
+      "step": 3340
+    },
+    {
+      "epoch": 1.662531017369727,
+      "grad_norm": 0.022349951788783073,
+      "learning_rate": 1.3351861042183625e-05,
+      "loss": 0.1361,
+      "step": 3350
+    },
+    {
+      "epoch": 1.6674937965260546,
+      "grad_norm": 4.284688472747803,
+      "learning_rate": 1.3332009925558314e-05,
+      "loss": 0.0903,
+      "step": 3360
+    },
+    {
+      "epoch": 1.6724565756823822,
+      "grad_norm": 0.08580008149147034,
+      "learning_rate": 1.3312158808933005e-05,
+      "loss": 0.004,
+      "step": 3370
+    },
+    {
+      "epoch": 1.6774193548387095,
+      "grad_norm": 0.04226774722337723,
+      "learning_rate": 1.3292307692307692e-05,
+      "loss": 0.0053,
+      "step": 3380
+    },
+    {
+      "epoch": 1.6823821339950373,
+      "grad_norm": 0.02099980227649212,
+      "learning_rate": 1.3272456575682383e-05,
+      "loss": 0.0261,
+      "step": 3390
+    },
+    {
+      "epoch": 1.6873449131513647,
+      "grad_norm": 0.025450438261032104,
+      "learning_rate": 1.3252605459057073e-05,
+      "loss": 0.0015,
+      "step": 3400
+    },
+    {
+      "epoch": 1.6923076923076923,
+      "grad_norm": 0.016049442812800407,
+      "learning_rate": 1.3232754342431764e-05,
+      "loss": 0.0091,
+      "step": 3410
+    },
+    {
+      "epoch": 1.6972704714640199,
+      "grad_norm": 0.02252548560500145,
+      "learning_rate": 1.3212903225806453e-05,
+      "loss": 0.042,
+      "step": 3420
+    },
+    {
+      "epoch": 1.7022332506203472,
+      "grad_norm": 0.015311924740672112,
+      "learning_rate": 1.3193052109181142e-05,
+      "loss": 0.0435,
+      "step": 3430
+    },
+    {
+      "epoch": 1.707196029776675,
+      "grad_norm": 0.17135725915431976,
+      "learning_rate": 1.3173200992555833e-05,
+      "loss": 0.0139,
+      "step": 3440
+    },
+    {
+      "epoch": 1.7121588089330024,
+      "grad_norm": 0.04031744971871376,
+      "learning_rate": 1.315334987593052e-05,
+      "loss": 0.022,
+      "step": 3450
+    },
+    {
+      "epoch": 1.71712158808933,
+      "grad_norm": 0.02933545410633087,
+      "learning_rate": 1.3133498759305211e-05,
+      "loss": 0.0036,
+      "step": 3460
+    },
+    {
+      "epoch": 1.7220843672456576,
+      "grad_norm": 1.0902290344238281,
+      "learning_rate": 1.31136476426799e-05,
+      "loss": 0.0873,
+      "step": 3470
+    },
+    {
+      "epoch": 1.727047146401985,
+      "grad_norm": 0.018781030550599098,
+      "learning_rate": 1.3093796526054591e-05,
+      "loss": 0.0296,
+      "step": 3480
+    },
+    {
+      "epoch": 1.7320099255583128,
+      "grad_norm": 0.01297234371304512,
+      "learning_rate": 1.307394540942928e-05,
+      "loss": 0.0081,
+      "step": 3490
+    },
+    {
+      "epoch": 1.7369727047146402,
+      "grad_norm": 1.0218632221221924,
+      "learning_rate": 1.3054094292803972e-05,
+      "loss": 0.0631,
+      "step": 3500
+    },
+    {
+      "epoch": 1.7419354838709677,
+      "grad_norm": 0.012826375663280487,
+      "learning_rate": 1.303424317617866e-05,
+      "loss": 0.0023,
+      "step": 3510
+    },
+    {
+      "epoch": 1.7468982630272953,
+      "grad_norm": 0.019672391936182976,
+      "learning_rate": 1.3014392059553352e-05,
+      "loss": 0.0103,
+      "step": 3520
+    },
+    {
+      "epoch": 1.7518610421836227,
+      "grad_norm": 0.20490695536136627,
+      "learning_rate": 1.299454094292804e-05,
+      "loss": 0.0101,
+      "step": 3530
+    },
+    {
+      "epoch": 1.7568238213399505,
+      "grad_norm": 0.061675168573856354,
+      "learning_rate": 1.297468982630273e-05,
+      "loss": 0.1038,
+      "step": 3540
+    },
+    {
+      "epoch": 1.7617866004962779,
+      "grad_norm": 0.028713202103972435,
+      "learning_rate": 1.295483870967742e-05,
+      "loss": 0.0312,
+      "step": 3550
+    },
+    {
+      "epoch": 1.7667493796526055,
+      "grad_norm": 12.863529205322266,
+      "learning_rate": 1.293498759305211e-05,
+      "loss": 0.0733,
+      "step": 3560
+    },
+    {
+      "epoch": 1.771712158808933,
+      "grad_norm": 24.391372680664062,
+      "learning_rate": 1.29151364764268e-05,
+      "loss": 0.1042,
+      "step": 3570
+    },
+    {
+      "epoch": 1.7766749379652604,
+      "grad_norm": 0.01382366195321083,
+      "learning_rate": 1.289528535980149e-05,
+      "loss": 0.0284,
+      "step": 3580
+    },
+    {
+      "epoch": 1.7816377171215882,
+      "grad_norm": 0.019975418224930763,
+      "learning_rate": 1.287543424317618e-05,
+      "loss": 0.0012,
+      "step": 3590
+    },
+    {
+      "epoch": 1.7866004962779156,
+      "grad_norm": 0.007871775887906551,
+      "learning_rate": 1.285558312655087e-05,
+      "loss": 0.1208,
+      "step": 3600
+    },
+    {
+      "epoch": 1.7915632754342432,
+      "grad_norm": 0.019381511956453323,
+      "learning_rate": 1.2835732009925558e-05,
+      "loss": 0.0046,
+      "step": 3610
+    },
+    {
+      "epoch": 1.7965260545905708,
+      "grad_norm": 0.013670637272298336,
+      "learning_rate": 1.2815880893300249e-05,
+      "loss": 0.0546,
+      "step": 3620
+    },
+    {
+      "epoch": 1.8014888337468982,
+      "grad_norm": 0.04148397967219353,
+      "learning_rate": 1.2796029776674938e-05,
+      "loss": 0.0383,
+      "step": 3630
+    },
+    {
+      "epoch": 1.8064516129032258,
+      "grad_norm": 0.029673762619495392,
+      "learning_rate": 1.277617866004963e-05,
+      "loss": 0.1479,
+      "step": 3640
+    },
+    {
+      "epoch": 1.8114143920595533,
+      "grad_norm": 0.013847707770764828,
+      "learning_rate": 1.2756327543424318e-05,
+      "loss": 0.1445,
+      "step": 3650
+    },
+    {
+      "epoch": 1.816377171215881,
+      "grad_norm": 0.02659500390291214,
+      "learning_rate": 1.273647642679901e-05,
+      "loss": 0.0098,
+      "step": 3660
+    },
+    {
+      "epoch": 1.8213399503722085,
+      "grad_norm": 0.05986494943499565,
+      "learning_rate": 1.2716625310173699e-05,
+      "loss": 0.1771,
+      "step": 3670
+    },
+    {
+      "epoch": 1.826302729528536,
+      "grad_norm": 0.04136352613568306,
+      "learning_rate": 1.269677419354839e-05,
+      "loss": 0.0054,
+      "step": 3680
+    },
+    {
+      "epoch": 1.8312655086848635,
+      "grad_norm": 0.15637284517288208,
+      "learning_rate": 1.2676923076923077e-05,
+      "loss": 0.0506,
+      "step": 3690
+    },
+    {
+      "epoch": 1.836228287841191,
+      "grad_norm": 0.010170268826186657,
+      "learning_rate": 1.2657071960297768e-05,
+      "loss": 0.0489,
+      "step": 3700
+    },
+    {
+      "epoch": 1.8411910669975184,
+      "grad_norm": 0.01803481951355934,
+      "learning_rate": 1.2637220843672457e-05,
+      "loss": 0.1091,
+      "step": 3710
+    },
+    {
+      "epoch": 1.8461538461538463,
+      "grad_norm": 0.014371192082762718,
+      "learning_rate": 1.2617369727047148e-05,
+      "loss": 0.0032,
+      "step": 3720
+    },
+    {
+      "epoch": 1.8511166253101736,
+      "grad_norm": 0.010766633786261082,
+      "learning_rate": 1.2597518610421837e-05,
+      "loss": 0.0161,
+      "step": 3730
+    },
+    {
+      "epoch": 1.8560794044665012,
+      "grad_norm": 0.0055402047000825405,
+      "learning_rate": 1.2577667493796528e-05,
+      "loss": 0.0011,
+      "step": 3740
+    },
+    {
+      "epoch": 1.8610421836228288,
+      "grad_norm": 4.953767776489258,
+      "learning_rate": 1.2557816377171217e-05,
+      "loss": 0.1624,
+      "step": 3750
+    },
+    {
+      "epoch": 1.8660049627791562,
+      "grad_norm": 0.0288150105625391,
+      "learning_rate": 1.2537965260545905e-05,
+      "loss": 0.0129,
+      "step": 3760
+    },
+    {
+      "epoch": 1.870967741935484,
+      "grad_norm": 9.697022438049316,
+      "learning_rate": 1.2518114143920596e-05,
+      "loss": 0.1279,
+      "step": 3770
+    },
+    {
+      "epoch": 1.8759305210918114,
+      "grad_norm": 0.04901479184627533,
+      "learning_rate": 1.2498263027295285e-05,
+      "loss": 0.1341,
+      "step": 3780
+    },
+    {
+      "epoch": 1.880893300248139,
+      "grad_norm": 0.03689178079366684,
+      "learning_rate": 1.2478411910669976e-05,
+      "loss": 0.0377,
+      "step": 3790
+    },
+    {
+      "epoch": 1.8858560794044665,
+      "grad_norm": 0.9575324058532715,
+      "learning_rate": 1.2458560794044665e-05,
+      "loss": 0.0089,
+      "step": 3800
+    },
+    {
+      "epoch": 1.890818858560794,
+      "grad_norm": 0.010843999683856964,
+      "learning_rate": 1.2438709677419356e-05,
+      "loss": 0.1107,
+      "step": 3810
+    },
+    {
+      "epoch": 1.8957816377171217,
+      "grad_norm": 0.022446051239967346,
+      "learning_rate": 1.2418858560794045e-05,
+      "loss": 0.0457,
+      "step": 3820
+    },
+    {
+      "epoch": 1.900744416873449,
+      "grad_norm": 9.969408988952637,
+      "learning_rate": 1.2399007444168736e-05,
+      "loss": 0.044,
+      "step": 3830
+    },
+    {
+      "epoch": 1.9057071960297767,
+      "grad_norm": 0.07644571363925934,
+      "learning_rate": 1.2379156327543424e-05,
+      "loss": 0.0042,
+      "step": 3840
+    },
+    {
+      "epoch": 1.9106699751861043,
+      "grad_norm": 4.885651111602783,
+      "learning_rate": 1.2359305210918115e-05,
+      "loss": 0.1051,
+      "step": 3850
+    },
+    {
+      "epoch": 1.9156327543424316,
+      "grad_norm": 0.010463743470609188,
+      "learning_rate": 1.2339454094292804e-05,
+      "loss": 0.0643,
+      "step": 3860
+    },
+    {
+      "epoch": 1.9205955334987594,
+      "grad_norm": 0.02377431094646454,
+      "learning_rate": 1.2319602977667495e-05,
+      "loss": 0.004,
+      "step": 3870
+    },
+    {
+      "epoch": 1.9255583126550868,
+      "grad_norm": 0.11802198737859726,
+      "learning_rate": 1.2299751861042184e-05,
+      "loss": 0.1587,
+      "step": 3880
+    },
+    {
+      "epoch": 1.9305210918114144,
+      "grad_norm": 0.02262243442237377,
+      "learning_rate": 1.2279900744416875e-05,
+      "loss": 0.0633,
+      "step": 3890
+    },
+    {
+      "epoch": 1.935483870967742,
+      "grad_norm": 0.011804407462477684,
+      "learning_rate": 1.2260049627791564e-05,
+      "loss": 0.1357,
+      "step": 3900
+    },
+    {
+      "epoch": 1.9404466501240694,
+      "grad_norm": 2.050931930541992,
+      "learning_rate": 1.2240198511166255e-05,
+      "loss": 0.1882,
+      "step": 3910
+    },
+    {
+      "epoch": 1.9454094292803972,
+      "grad_norm": 0.028718573972582817,
+      "learning_rate": 1.2220347394540943e-05,
+      "loss": 0.0456,
+      "step": 3920
+    },
+    {
+      "epoch": 1.9503722084367245,
+      "grad_norm": 0.19728246331214905,
+      "learning_rate": 1.2200496277915634e-05,
+      "loss": 0.0862,
+      "step": 3930
+    },
+    {
+      "epoch": 1.9553349875930521,
+      "grad_norm": 0.057744573801755905,
+      "learning_rate": 1.2180645161290323e-05,
+      "loss": 0.0009,
+      "step": 3940
+    },
+    {
+      "epoch": 1.9602977667493797,
+      "grad_norm": 0.824889600276947,
+      "learning_rate": 1.2160794044665014e-05,
+      "loss": 0.1256,
+      "step": 3950
+    },
+    {
+      "epoch": 1.965260545905707,
+      "grad_norm": 0.25211405754089355,
+      "learning_rate": 1.2140942928039703e-05,
+      "loss": 0.035,
+      "step": 3960
+    },
+    {
+      "epoch": 1.970223325062035,
+      "grad_norm": 0.6483780145645142,
+      "learning_rate": 1.2121091811414394e-05,
+      "loss": 0.0189,
+      "step": 3970
+    },
+    {
+      "epoch": 1.9751861042183623,
+      "grad_norm": 0.1307402402162552,
+      "learning_rate": 1.2101240694789083e-05,
+      "loss": 0.0522,
+      "step": 3980
+    },
+    {
+      "epoch": 1.9801488833746899,
+      "grad_norm": 0.0058226073160767555,
+      "learning_rate": 1.2081389578163774e-05,
+      "loss": 0.1483,
+      "step": 3990
+    },
+    {
+      "epoch": 1.9851116625310175,
+      "grad_norm": 0.011639786884188652,
+      "learning_rate": 1.2061538461538462e-05,
+      "loss": 0.0036,
+      "step": 4000
+    },
+    {
+      "epoch": 1.9900744416873448,
+      "grad_norm": 0.9634393453598022,
+      "learning_rate": 1.2041687344913152e-05,
+      "loss": 0.0767,
+      "step": 4010
+    },
+    {
+      "epoch": 1.9950372208436724,
+      "grad_norm": 0.015393534675240517,
+      "learning_rate": 1.2021836228287842e-05,
+      "loss": 0.0206,
+      "step": 4020
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.01411303412169218,
+      "learning_rate": 1.2001985111662533e-05,
+      "loss": 0.008,
+      "step": 4030
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 10075,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 5,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4356984445440000.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-4030/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:859efea2201c1469208153e54e3977a99dc6a369e5e443e74e5277d4b63a6e19
+size 5841

checkpoint-6045/README.md

@@ -0,0 +1,206 @@
+---
+base_model: protectai/deberta-v3-base-prompt-injection-v2
+library_name: peft
+tags:
+- base_model:adapter:protectai/deberta-v3-base-prompt-injection-v2
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

checkpoint-6045/adapter_config.json

@@ -0,0 +1,44 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "protectai/deberta-v3-base-prompt-injection-v2",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "value_proj",
+    "query_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-6045/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c93da9750f2b7749b57fb6dc1b2277a443b58c89c46cc4dc577adafe665bab0
+size 9450704

checkpoint-6045/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5997dd5ea83bf723521dd93eb7347baa17f87e36f61d040ad7d3d08bbe7b3824
+size 18930123

checkpoint-6045/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18ee72212a030d8452a57c57be98ab1e17510937f683e62d5ee2c344040f04c2
+size 14645

checkpoint-6045/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7207a60515ba26ba02a6e3b66fb20ca760789c077bc0a7b95de2554829d36af
+size 1383

checkpoint-6045/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ab71b57431a604c98e8d67cfcc90735fb8211c2acb9d8ef158fed260c91c089
+size 1465

checkpoint-6045/special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-6045/tokenizer_config.json

@@ -0,0 +1,63 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128000": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "split_by_punct": false,
+  "stride": 0,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}

checkpoint-6045/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:859efea2201c1469208153e54e3977a99dc6a369e5e443e74e5277d4b63a6e19
+size 5841