Upload folder using huggingface_hub

README.md

@@ -0,0 +1,207 @@
+---
+base_model: distilgpt2
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:distilgpt2
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.0

adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "distilgpt2",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_proj",
+    "c_attn"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3efcec786a7e5e5420f5989a58a6947f312a4c8a0a88d0c1f2a9044c040bb261
+size 3248616

checkpoint-1000/README.md

@@ -0,0 +1,207 @@
+---
+base_model: distilgpt2
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:distilgpt2
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.0

checkpoint-1000/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "distilgpt2",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_proj",
+    "c_attn"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e8ee6d4fa553a2205fc2ed4b4c0689a9f4326d5bb00e3141f133960c2b8e19a
+size 3248616

checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37207bf789a542293a103be9ff481f7bb2af2224225965f624ac804e8b2e1003
+size 6518714

checkpoint-1000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ef8e0419b49b0ef5350645fe4a8fecb5ac089d0189be793fc01e31fb99505eb
+size 14244

checkpoint-1000/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d8fdcd0311eba9854fff738038ed4c1a269832665b4d88ba4e4e3d02a1a7e0e
+size 988

checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72668d829dbdb7f9fcf02b26ca0fd56bd5ec9500d2675771791441d2766c2e2c
+size 1064

checkpoint-1000/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1000/tokenizer_config.json

@@ -0,0 +1,28 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "max_length": 180,
+  "model_max_length": 1024,
+  "pad_to_multiple_of": null,
+  "pad_token": "<|endoftext|>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "stride": 0,
+  "tokenizer_class": "GPT2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "<|endoftext|>"
+}

checkpoint-1000/trainer_state.json

@@ -0,0 +1,85 @@
+{
+  "best_global_step": 1000,
+  "best_metric": 0.9456237554550171,
+  "best_model_checkpoint": "./banking-sms-parser-v10/checkpoint-1000",
+  "epoch": 0.2962962962962963,
+  "eval_steps": 500,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.05925925925925926,
+      "grad_norm": 0.47084560990333557,
+      "learning_rate": 0.0002955777777777778,
+      "loss": 2.0436,
+      "step": 200
+    },
+    {
+      "epoch": 0.11851851851851852,
+      "grad_norm": 0.40365636348724365,
+      "learning_rate": 0.0002911333333333333,
+      "loss": 1.3594,
+      "step": 400
+    },
+    {
+      "epoch": 0.14814814814814814,
+      "eval_loss": 1.1683876514434814,
+      "eval_runtime": 29.5095,
+      "eval_samples_per_second": 101.662,
+      "eval_steps_per_second": 12.708,
+      "step": 500
+    },
+    {
+      "epoch": 0.17777777777777778,
+      "grad_norm": 0.5327613353729248,
+      "learning_rate": 0.0002866888888888889,
+      "loss": 1.2262,
+      "step": 600
+    },
+    {
+      "epoch": 0.23703703703703705,
+      "grad_norm": 0.5959504246711731,
+      "learning_rate": 0.0002822444444444444,
+      "loss": 1.1381,
+      "step": 800
+    },
+    {
+      "epoch": 0.2962962962962963,
+      "grad_norm": 0.4996171295642853,
+      "learning_rate": 0.0002778,
+      "loss": 1.066,
+      "step": 1000
+    },
+    {
+      "epoch": 0.2962962962962963,
+      "eval_loss": 0.9456237554550171,
+      "eval_runtime": 29.4299,
+      "eval_samples_per_second": 101.937,
+      "eval_steps_per_second": 12.742,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 200,
+  "max_steps": 13500,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 4,
+  "save_steps": 1000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 665698959360000.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5b2e2a527ba622332dbb2398486338b6fc135483c9c1c937ef5cefadf332325
+size 5304

checkpoint-10000/README.md

@@ -0,0 +1,207 @@
+---
+base_model: distilgpt2
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:distilgpt2
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.0

checkpoint-10000/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "distilgpt2",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_proj",
+    "c_attn"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-10000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e469b5b67b9cfb1e04ccb7dfc3e2a21c24e5d78f4240d8b68a775f4b8603fe01
+size 3248616

checkpoint-10000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b446e97e2a7c87bc3aad2815aa8460f724773ccd21cdd0d5511c7d66aaa56ef4
+size 6518714

checkpoint-10000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:281ee8a77a8e6d9db96e3a879e0288c8233833642fc0cf3abcf042aa7b35e32b
+size 14244

checkpoint-10000/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c072a285b6f4670d281023d2fdc827b249e4a784e3ebc49cfbe50a54914c99b
+size 988

checkpoint-10000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7173c7464031a760641151afa8096faf47fe8bf592067a6c5fec7ca4e6d8ec9a
+size 1064

checkpoint-10000/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-10000/tokenizer_config.json

@@ -0,0 +1,28 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "max_length": 180,
+  "model_max_length": 1024,
+  "pad_to_multiple_of": null,
+  "pad_token": "<|endoftext|>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "stride": 0,
+  "tokenizer_class": "GPT2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "<|endoftext|>"
+}

checkpoint-10000/trainer_state.json

@@ -0,0 +1,544 @@
+{
+  "best_global_step": 10000,
+  "best_metric": 0.46695125102996826,
+  "best_model_checkpoint": "./banking-sms-parser-v10/checkpoint-10000",
+  "epoch": 2.962962962962963,
+  "eval_steps": 500,
+  "global_step": 10000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.05925925925925926,
+      "grad_norm": 0.47084560990333557,
+      "learning_rate": 0.0002955777777777778,
+      "loss": 2.0436,
+      "step": 200
+    },
+    {
+      "epoch": 0.11851851851851852,
+      "grad_norm": 0.40365636348724365,
+      "learning_rate": 0.0002911333333333333,
+      "loss": 1.3594,
+      "step": 400
+    },
+    {
+      "epoch": 0.14814814814814814,
+      "eval_loss": 1.1683876514434814,
+      "eval_runtime": 29.5095,
+      "eval_samples_per_second": 101.662,
+      "eval_steps_per_second": 12.708,
+      "step": 500
+    },
+    {
+      "epoch": 0.17777777777777778,
+      "grad_norm": 0.5327613353729248,
+      "learning_rate": 0.0002866888888888889,
+      "loss": 1.2262,
+      "step": 600
+    },
+    {
+      "epoch": 0.23703703703703705,
+      "grad_norm": 0.5959504246711731,
+      "learning_rate": 0.0002822444444444444,
+      "loss": 1.1381,
+      "step": 800
+    },
+    {
+      "epoch": 0.2962962962962963,
+      "grad_norm": 0.4996171295642853,
+      "learning_rate": 0.0002778,
+      "loss": 1.066,
+      "step": 1000
+    },
+    {
+      "epoch": 0.2962962962962963,
+      "eval_loss": 0.9456237554550171,
+      "eval_runtime": 29.4299,
+      "eval_samples_per_second": 101.937,
+      "eval_steps_per_second": 12.742,
+      "step": 1000
+    },
+    {
+      "epoch": 0.35555555555555557,
+      "grad_norm": 0.47130197286605835,
+      "learning_rate": 0.0002733555555555555,
+      "loss": 1.0067,
+      "step": 1200
+    },
+    {
+      "epoch": 0.4148148148148148,
+      "grad_norm": 0.693696916103363,
+      "learning_rate": 0.0002689111111111111,
+      "loss": 0.969,
+      "step": 1400
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "eval_loss": 0.8003639578819275,
+      "eval_runtime": 29.3553,
+      "eval_samples_per_second": 102.196,
+      "eval_steps_per_second": 12.775,
+      "step": 1500
+    },
+    {
+      "epoch": 0.4740740740740741,
+      "grad_norm": 0.5786986351013184,
+      "learning_rate": 0.0002644666666666666,
+      "loss": 0.9081,
+      "step": 1600
+    },
+    {
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.5209688544273376,
+      "learning_rate": 0.0002600222222222222,
+      "loss": 0.8801,
+      "step": 1800
+    },
+    {
+      "epoch": 0.5925925925925926,
+      "grad_norm": 0.7497162818908691,
+      "learning_rate": 0.0002555777777777778,
+      "loss": 0.8183,
+      "step": 2000
+    },
+    {
+      "epoch": 0.5925925925925926,
+      "eval_loss": 0.7105592489242554,
+      "eval_runtime": 29.7273,
+      "eval_samples_per_second": 100.917,
+      "eval_steps_per_second": 12.615,
+      "step": 2000
+    },
+    {
+      "epoch": 0.6518518518518519,
+      "grad_norm": 0.7358675599098206,
+      "learning_rate": 0.00025113333333333333,
+      "loss": 0.809,
+      "step": 2200
+    },
+    {
+      "epoch": 0.7111111111111111,
+      "grad_norm": 0.5220760107040405,
+      "learning_rate": 0.0002466888888888889,
+      "loss": 0.7799,
+      "step": 2400
+    },
+    {
+      "epoch": 0.7407407407407407,
+      "eval_loss": 0.6572133898735046,
+      "eval_runtime": 29.3661,
+      "eval_samples_per_second": 102.158,
+      "eval_steps_per_second": 12.77,
+      "step": 2500
+    },
+    {
+      "epoch": 0.7703703703703704,
+      "grad_norm": 0.4607163965702057,
+      "learning_rate": 0.00024224444444444443,
+      "loss": 0.7557,
+      "step": 2600
+    },
+    {
+      "epoch": 0.8296296296296296,
+      "grad_norm": 0.5450932383537292,
+      "learning_rate": 0.00023779999999999998,
+      "loss": 0.7264,
+      "step": 2800
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 0.5149521231651306,
+      "learning_rate": 0.00023335555555555556,
+      "loss": 0.7054,
+      "step": 3000
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "eval_loss": 0.6189373731613159,
+      "eval_runtime": 29.6913,
+      "eval_samples_per_second": 101.04,
+      "eval_steps_per_second": 12.63,
+      "step": 3000
+    },
+    {
+      "epoch": 0.9481481481481482,
+      "grad_norm": 0.5345056056976318,
+      "learning_rate": 0.0002289111111111111,
+      "loss": 0.7026,
+      "step": 3200
+    },
+    {
+      "epoch": 1.0074074074074073,
+      "grad_norm": 0.5385560989379883,
+      "learning_rate": 0.00022446666666666666,
+      "loss": 0.6888,
+      "step": 3400
+    },
+    {
+      "epoch": 1.037037037037037,
+      "eval_loss": 0.5899662971496582,
+      "eval_runtime": 29.4218,
+      "eval_samples_per_second": 101.965,
+      "eval_steps_per_second": 12.746,
+      "step": 3500
+    },
+    {
+      "epoch": 1.0666666666666667,
+      "grad_norm": 0.5609071850776672,
+      "learning_rate": 0.0002200222222222222,
+      "loss": 0.6671,
+      "step": 3600
+    },
+    {
+      "epoch": 1.125925925925926,
+      "grad_norm": 0.5616484880447388,
+      "learning_rate": 0.00021557777777777776,
+      "loss": 0.6636,
+      "step": 3800
+    },
+    {
+      "epoch": 1.1851851851851851,
+      "grad_norm": 0.5596534609794617,
+      "learning_rate": 0.0002111333333333333,
+      "loss": 0.6515,
+      "step": 4000
+    },
+    {
+      "epoch": 1.1851851851851851,
+      "eval_loss": 0.5668333172798157,
+      "eval_runtime": 29.3385,
+      "eval_samples_per_second": 102.255,
+      "eval_steps_per_second": 12.782,
+      "step": 4000
+    },
+    {
+      "epoch": 1.2444444444444445,
+      "grad_norm": 0.5900132656097412,
+      "learning_rate": 0.00020668888888888888,
+      "loss": 0.6409,
+      "step": 4200
+    },
+    {
+      "epoch": 1.3037037037037038,
+      "grad_norm": 0.6175922751426697,
+      "learning_rate": 0.00020224444444444443,
+      "loss": 0.641,
+      "step": 4400
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "eval_loss": 0.5480825304985046,
+      "eval_runtime": 29.457,
+      "eval_samples_per_second": 101.843,
+      "eval_steps_per_second": 12.73,
+      "step": 4500
+    },
+    {
+      "epoch": 1.362962962962963,
+      "grad_norm": 0.42820265889167786,
+      "learning_rate": 0.00019779999999999998,
+      "loss": 0.6212,
+      "step": 4600
+    },
+    {
+      "epoch": 1.4222222222222223,
+      "grad_norm": 0.7179502248764038,
+      "learning_rate": 0.00019335555555555553,
+      "loss": 0.6268,
+      "step": 4800
+    },
+    {
+      "epoch": 1.4814814814814814,
+      "grad_norm": 0.4201723635196686,
+      "learning_rate": 0.00018891111111111108,
+      "loss": 0.6083,
+      "step": 5000
+    },
+    {
+      "epoch": 1.4814814814814814,
+      "eval_loss": 0.538067638874054,
+      "eval_runtime": 29.3858,
+      "eval_samples_per_second": 102.09,
+      "eval_steps_per_second": 12.761,
+      "step": 5000
+    },
+    {
+      "epoch": 1.5407407407407407,
+      "grad_norm": 0.6730241775512695,
+      "learning_rate": 0.00018446666666666666,
+      "loss": 0.6148,
+      "step": 5200
+    },
+    {
+      "epoch": 1.6,
+      "grad_norm": 0.5300161242485046,
+      "learning_rate": 0.0001800222222222222,
+      "loss": 0.6001,
+      "step": 5400
+    },
+    {
+      "epoch": 1.6296296296296298,
+      "eval_loss": 0.5220749378204346,
+      "eval_runtime": 29.4236,
+      "eval_samples_per_second": 101.959,
+      "eval_steps_per_second": 12.745,
+      "step": 5500
+    },
+    {
+      "epoch": 1.6592592592592592,
+      "grad_norm": 0.45725536346435547,
+      "learning_rate": 0.00017557777777777776,
+      "loss": 0.605,
+      "step": 5600
+    },
+    {
+      "epoch": 1.7185185185185186,
+      "grad_norm": 0.49325183033943176,
+      "learning_rate": 0.00017115555555555554,
+      "loss": 0.6024,
+      "step": 5800
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "grad_norm": 0.5796788930892944,
+      "learning_rate": 0.00016671111111111109,
+      "loss": 0.5941,
+      "step": 6000
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "eval_loss": 0.5122686624526978,
+      "eval_runtime": 29.5133,
+      "eval_samples_per_second": 101.649,
+      "eval_steps_per_second": 12.706,
+      "step": 6000
+    },
+    {
+      "epoch": 1.837037037037037,
+      "grad_norm": 0.5793591737747192,
+      "learning_rate": 0.00016226666666666664,
+      "loss": 0.5809,
+      "step": 6200
+    },
+    {
+      "epoch": 1.8962962962962964,
+      "grad_norm": 0.4882641136646271,
+      "learning_rate": 0.00015782222222222219,
+      "loss": 0.5869,
+      "step": 6400
+    },
+    {
+      "epoch": 1.925925925925926,
+      "eval_loss": 0.5018052458763123,
+      "eval_runtime": 29.2669,
+      "eval_samples_per_second": 102.505,
+      "eval_steps_per_second": 12.813,
+      "step": 6500
+    },
+    {
+      "epoch": 1.9555555555555557,
+      "grad_norm": 0.5069060325622559,
+      "learning_rate": 0.0001533777777777778,
+      "loss": 0.5812,
+      "step": 6600
+    },
+    {
+      "epoch": 2.0148148148148146,
+      "grad_norm": 0.44775187969207764,
+      "learning_rate": 0.00014893333333333334,
+      "loss": 0.5397,
+      "step": 6800
+    },
+    {
+      "epoch": 2.074074074074074,
+      "grad_norm": 0.5677600502967834,
+      "learning_rate": 0.0001444888888888889,
+      "loss": 0.5567,
+      "step": 7000
+    },
+    {
+      "epoch": 2.074074074074074,
+      "eval_loss": 0.49609723687171936,
+      "eval_runtime": 29.4825,
+      "eval_samples_per_second": 101.755,
+      "eval_steps_per_second": 12.719,
+      "step": 7000
+    },
+    {
+      "epoch": 2.1333333333333333,
+      "grad_norm": 0.5369960069656372,
+      "learning_rate": 0.00014004444444444444,
+      "loss": 0.5591,
+      "step": 7200
+    },
+    {
+      "epoch": 2.1925925925925926,
+      "grad_norm": 0.353333055973053,
+      "learning_rate": 0.0001356,
+      "loss": 0.5717,
+      "step": 7400
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "eval_loss": 0.49044421315193176,
+      "eval_runtime": 29.573,
+      "eval_samples_per_second": 101.444,
+      "eval_steps_per_second": 12.68,
+      "step": 7500
+    },
+    {
+      "epoch": 2.251851851851852,
+      "grad_norm": 0.45505258440971375,
+      "learning_rate": 0.00013115555555555554,
+      "loss": 0.5576,
+      "step": 7600
+    },
+    {
+      "epoch": 2.311111111111111,
+      "grad_norm": 0.4602493643760681,
+      "learning_rate": 0.00012671111111111112,
+      "loss": 0.5407,
+      "step": 7800
+    },
+    {
+      "epoch": 2.3703703703703702,
+      "grad_norm": 0.4852631688117981,
+      "learning_rate": 0.00012226666666666667,
+      "loss": 0.5523,
+      "step": 8000
+    },
+    {
+      "epoch": 2.3703703703703702,
+      "eval_loss": 0.48393580317497253,
+      "eval_runtime": 29.3448,
+      "eval_samples_per_second": 102.233,
+      "eval_steps_per_second": 12.779,
+      "step": 8000
+    },
+    {
+      "epoch": 2.4296296296296296,
+      "grad_norm": 0.4038563668727875,
+      "learning_rate": 0.0001178222222222222,
+      "loss": 0.5313,
+      "step": 8200
+    },
+    {
+      "epoch": 2.488888888888889,
+      "grad_norm": 0.5276423692703247,
+      "learning_rate": 0.00011339999999999999,
+      "loss": 0.5489,
+      "step": 8400
+    },
+    {
+      "epoch": 2.5185185185185186,
+      "eval_loss": 0.47875121235847473,
+      "eval_runtime": 29.4918,
+      "eval_samples_per_second": 101.723,
+      "eval_steps_per_second": 12.715,
+      "step": 8500
+    },
+    {
+      "epoch": 2.5481481481481483,
+      "grad_norm": 0.539804995059967,
+      "learning_rate": 0.00010895555555555554,
+      "loss": 0.5456,
+      "step": 8600
+    },
+    {
+      "epoch": 2.6074074074074076,
+      "grad_norm": 0.629550039768219,
+      "learning_rate": 0.0001045111111111111,
+      "loss": 0.5523,
+      "step": 8800
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 0.6091557741165161,
+      "learning_rate": 0.00010006666666666666,
+      "loss": 0.5358,
+      "step": 9000
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "eval_loss": 0.4742780327796936,
+      "eval_runtime": 29.4044,
+      "eval_samples_per_second": 102.026,
+      "eval_steps_per_second": 12.753,
+      "step": 9000
+    },
+    {
+      "epoch": 2.725925925925926,
+      "grad_norm": 0.5220003724098206,
+      "learning_rate": 9.56222222222222e-05,
+      "loss": 0.5398,
+      "step": 9200
+    },
+    {
+      "epoch": 2.785185185185185,
+      "grad_norm": 0.4543071389198303,
+      "learning_rate": 9.117777777777777e-05,
+      "loss": 0.53,
+      "step": 9400
+    },
+    {
+      "epoch": 2.814814814814815,
+      "eval_loss": 0.4698003828525543,
+      "eval_runtime": 29.5022,
+      "eval_samples_per_second": 101.687,
+      "eval_steps_per_second": 12.711,
+      "step": 9500
+    },
+    {
+      "epoch": 2.8444444444444446,
+      "grad_norm": 0.4849318265914917,
+      "learning_rate": 8.673333333333332e-05,
+      "loss": 0.5278,
+      "step": 9600
+    },
+    {
+      "epoch": 2.9037037037037035,
+      "grad_norm": 0.40453964471817017,
+      "learning_rate": 8.228888888888888e-05,
+      "loss": 0.5448,
+      "step": 9800
+    },
+    {
+      "epoch": 2.962962962962963,
+      "grad_norm": 0.5250543355941772,
+      "learning_rate": 7.784444444444443e-05,
+      "loss": 0.538,
+      "step": 10000
+    },
+    {
+      "epoch": 2.962962962962963,
+      "eval_loss": 0.46695125102996826,
+      "eval_runtime": 29.5114,
+      "eval_samples_per_second": 101.656,
+      "eval_steps_per_second": 12.707,
+      "step": 10000
+    }
+  ],
+  "logging_steps": 200,
+  "max_steps": 13500,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 4,
+  "save_steps": 1000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6656989593600000.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-10000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5b2e2a527ba622332dbb2398486338b6fc135483c9c1c937ef5cefadf332325
+size 5304

checkpoint-11000/README.md

@@ -0,0 +1,207 @@
+---
+base_model: distilgpt2
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:distilgpt2
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.0

checkpoint-11000/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "distilgpt2",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_proj",
+    "c_attn"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-11000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:584793f3f1527007903559128e2fe5b814b9b5f4891a3af426913102ce2eb4af
+size 3248616

checkpoint-11000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9751901d1352fac6f2e365d2202a3dc2d03e11f9562f8ab51b7008943368a95
+size 6518714

checkpoint-11000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:963ca21a536f8365ddeacfa23786fde574fe723ceac54374eed1c1ee982bb3ee
+size 14244

checkpoint-11000/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93eff9e1f9576ce4414c6b4d8adfd7479e9fd6585ac64d4041c7aeec38be28c7
+size 988

checkpoint-11000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7aa624420f96a2e9654a0864e1ec4d155282f0b0eacfc3ce38b8b5bbbf1b8d57
+size 1064

checkpoint-11000/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-11000/tokenizer_config.json

@@ -0,0 +1,28 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "max_length": 180,
+  "model_max_length": 1024,
+  "pad_to_multiple_of": null,
+  "pad_token": "<|endoftext|>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "stride": 0,
+  "tokenizer_class": "GPT2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "<|endoftext|>"
+}

checkpoint-11000/trainer_state.json

@@ -0,0 +1,595 @@
+{
+  "best_global_step": 11000,
+  "best_metric": 0.4608609676361084,
+  "best_model_checkpoint": "./banking-sms-parser-v10/checkpoint-11000",
+  "epoch": 3.259259259259259,
+  "eval_steps": 500,
+  "global_step": 11000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.05925925925925926,
+      "grad_norm": 0.47084560990333557,
+      "learning_rate": 0.0002955777777777778,
+      "loss": 2.0436,
+      "step": 200
+    },
+    {
+      "epoch": 0.11851851851851852,
+      "grad_norm": 0.40365636348724365,
+      "learning_rate": 0.0002911333333333333,
+      "loss": 1.3594,
+      "step": 400
+    },
+    {
+      "epoch": 0.14814814814814814,
+      "eval_loss": 1.1683876514434814,
+      "eval_runtime": 29.5095,
+      "eval_samples_per_second": 101.662,
+      "eval_steps_per_second": 12.708,
+      "step": 500
+    },
+    {
+      "epoch": 0.17777777777777778,
+      "grad_norm": 0.5327613353729248,
+      "learning_rate": 0.0002866888888888889,
+      "loss": 1.2262,
+      "step": 600
+    },
+    {
+      "epoch": 0.23703703703703705,
+      "grad_norm": 0.5959504246711731,
+      "learning_rate": 0.0002822444444444444,
+      "loss": 1.1381,
+      "step": 800
+    },
+    {
+      "epoch": 0.2962962962962963,
+      "grad_norm": 0.4996171295642853,
+      "learning_rate": 0.0002778,
+      "loss": 1.066,
+      "step": 1000
+    },
+    {
+      "epoch": 0.2962962962962963,
+      "eval_loss": 0.9456237554550171,
+      "eval_runtime": 29.4299,
+      "eval_samples_per_second": 101.937,
+      "eval_steps_per_second": 12.742,
+      "step": 1000
+    },
+    {
+      "epoch": 0.35555555555555557,
+      "grad_norm": 0.47130197286605835,
+      "learning_rate": 0.0002733555555555555,
+      "loss": 1.0067,
+      "step": 1200
+    },
+    {
+      "epoch": 0.4148148148148148,
+      "grad_norm": 0.693696916103363,
+      "learning_rate": 0.0002689111111111111,
+      "loss": 0.969,
+      "step": 1400
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "eval_loss": 0.8003639578819275,
+      "eval_runtime": 29.3553,
+      "eval_samples_per_second": 102.196,
+      "eval_steps_per_second": 12.775,
+      "step": 1500
+    },
+    {
+      "epoch": 0.4740740740740741,
+      "grad_norm": 0.5786986351013184,
+      "learning_rate": 0.0002644666666666666,
+      "loss": 0.9081,
+      "step": 1600
+    },
+    {
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.5209688544273376,
+      "learning_rate": 0.0002600222222222222,
+      "loss": 0.8801,
+      "step": 1800
+    },
+    {
+      "epoch": 0.5925925925925926,
+      "grad_norm": 0.7497162818908691,
+      "learning_rate": 0.0002555777777777778,
+      "loss": 0.8183,
+      "step": 2000
+    },
+    {
+      "epoch": 0.5925925925925926,
+      "eval_loss": 0.7105592489242554,
+      "eval_runtime": 29.7273,
+      "eval_samples_per_second": 100.917,
+      "eval_steps_per_second": 12.615,
+      "step": 2000
+    },
+    {
+      "epoch": 0.6518518518518519,
+      "grad_norm": 0.7358675599098206,
+      "learning_rate": 0.00025113333333333333,
+      "loss": 0.809,
+      "step": 2200
+    },
+    {
+      "epoch": 0.7111111111111111,
+      "grad_norm": 0.5220760107040405,
+      "learning_rate": 0.0002466888888888889,
+      "loss": 0.7799,
+      "step": 2400
+    },
+    {
+      "epoch": 0.7407407407407407,
+      "eval_loss": 0.6572133898735046,
+      "eval_runtime": 29.3661,
+      "eval_samples_per_second": 102.158,
+      "eval_steps_per_second": 12.77,
+      "step": 2500
+    },
+    {
+      "epoch": 0.7703703703703704,
+      "grad_norm": 0.4607163965702057,
+      "learning_rate": 0.00024224444444444443,
+      "loss": 0.7557,
+      "step": 2600
+    },
+    {
+      "epoch": 0.8296296296296296,
+      "grad_norm": 0.5450932383537292,
+      "learning_rate": 0.00023779999999999998,
+      "loss": 0.7264,
+      "step": 2800
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 0.5149521231651306,
+      "learning_rate": 0.00023335555555555556,
+      "loss": 0.7054,
+      "step": 3000
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "eval_loss": 0.6189373731613159,
+      "eval_runtime": 29.6913,
+      "eval_samples_per_second": 101.04,
+      "eval_steps_per_second": 12.63,
+      "step": 3000
+    },
+    {
+      "epoch": 0.9481481481481482,
+      "grad_norm": 0.5345056056976318,
+      "learning_rate": 0.0002289111111111111,
+      "loss": 0.7026,
+      "step": 3200
+    },
+    {
+      "epoch": 1.0074074074074073,
+      "grad_norm": 0.5385560989379883,
+      "learning_rate": 0.00022446666666666666,
+      "loss": 0.6888,
+      "step": 3400
+    },
+    {
+      "epoch": 1.037037037037037,
+      "eval_loss": 0.5899662971496582,
+      "eval_runtime": 29.4218,
+      "eval_samples_per_second": 101.965,
+      "eval_steps_per_second": 12.746,
+      "step": 3500
+    },
+    {
+      "epoch": 1.0666666666666667,
+      "grad_norm": 0.5609071850776672,
+      "learning_rate": 0.0002200222222222222,
+      "loss": 0.6671,
+      "step": 3600
+    },
+    {
+      "epoch": 1.125925925925926,
+      "grad_norm": 0.5616484880447388,
+      "learning_rate": 0.00021557777777777776,
+      "loss": 0.6636,
+      "step": 3800
+    },
+    {
+      "epoch": 1.1851851851851851,
+      "grad_norm": 0.5596534609794617,
+      "learning_rate": 0.0002111333333333333,
+      "loss": 0.6515,
+      "step": 4000
+    },
+    {
+      "epoch": 1.1851851851851851,
+      "eval_loss": 0.5668333172798157,
+      "eval_runtime": 29.3385,
+      "eval_samples_per_second": 102.255,
+      "eval_steps_per_second": 12.782,
+      "step": 4000
+    },
+    {
+      "epoch": 1.2444444444444445,
+      "grad_norm": 0.5900132656097412,
+      "learning_rate": 0.00020668888888888888,
+      "loss": 0.6409,
+      "step": 4200
+    },
+    {
+      "epoch": 1.3037037037037038,
+      "grad_norm": 0.6175922751426697,
+      "learning_rate": 0.00020224444444444443,
+      "loss": 0.641,
+      "step": 4400
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "eval_loss": 0.5480825304985046,
+      "eval_runtime": 29.457,
+      "eval_samples_per_second": 101.843,
+      "eval_steps_per_second": 12.73,
+      "step": 4500
+    },
+    {
+      "epoch": 1.362962962962963,
+      "grad_norm": 0.42820265889167786,
+      "learning_rate": 0.00019779999999999998,
+      "loss": 0.6212,
+      "step": 4600
+    },
+    {
+      "epoch": 1.4222222222222223,
+      "grad_norm": 0.7179502248764038,
+      "learning_rate": 0.00019335555555555553,
+      "loss": 0.6268,
+      "step": 4800
+    },
+    {
+      "epoch": 1.4814814814814814,
+      "grad_norm": 0.4201723635196686,
+      "learning_rate": 0.00018891111111111108,
+      "loss": 0.6083,
+      "step": 5000
+    },
+    {
+      "epoch": 1.4814814814814814,
+      "eval_loss": 0.538067638874054,
+      "eval_runtime": 29.3858,
+      "eval_samples_per_second": 102.09,
+      "eval_steps_per_second": 12.761,
+      "step": 5000
+    },
+    {
+      "epoch": 1.5407407407407407,
+      "grad_norm": 0.6730241775512695,
+      "learning_rate": 0.00018446666666666666,
+      "loss": 0.6148,
+      "step": 5200
+    },
+    {
+      "epoch": 1.6,
+      "grad_norm": 0.5300161242485046,
+      "learning_rate": 0.0001800222222222222,
+      "loss": 0.6001,
+      "step": 5400
+    },
+    {
+      "epoch": 1.6296296296296298,
+      "eval_loss": 0.5220749378204346,
+      "eval_runtime": 29.4236,
+      "eval_samples_per_second": 101.959,
+      "eval_steps_per_second": 12.745,
+      "step": 5500
+    },
+    {
+      "epoch": 1.6592592592592592,
+      "grad_norm": 0.45725536346435547,
+      "learning_rate": 0.00017557777777777776,
+      "loss": 0.605,
+      "step": 5600
+    },
+    {
+      "epoch": 1.7185185185185186,
+      "grad_norm": 0.49325183033943176,
+      "learning_rate": 0.00017115555555555554,
+      "loss": 0.6024,
+      "step": 5800
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "grad_norm": 0.5796788930892944,
+      "learning_rate": 0.00016671111111111109,
+      "loss": 0.5941,
+      "step": 6000
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "eval_loss": 0.5122686624526978,
+      "eval_runtime": 29.5133,
+      "eval_samples_per_second": 101.649,
+      "eval_steps_per_second": 12.706,
+      "step": 6000
+    },
+    {
+      "epoch": 1.837037037037037,
+      "grad_norm": 0.5793591737747192,
+      "learning_rate": 0.00016226666666666664,
+      "loss": 0.5809,
+      "step": 6200
+    },
+    {
+      "epoch": 1.8962962962962964,
+      "grad_norm": 0.4882641136646271,
+      "learning_rate": 0.00015782222222222219,
+      "loss": 0.5869,
+      "step": 6400
+    },
+    {
+      "epoch": 1.925925925925926,
+      "eval_loss": 0.5018052458763123,
+      "eval_runtime": 29.2669,
+      "eval_samples_per_second": 102.505,
+      "eval_steps_per_second": 12.813,
+      "step": 6500
+    },
+    {
+      "epoch": 1.9555555555555557,
+      "grad_norm": 0.5069060325622559,
+      "learning_rate": 0.0001533777777777778,
+      "loss": 0.5812,
+      "step": 6600
+    },
+    {
+      "epoch": 2.0148148148148146,
+      "grad_norm": 0.44775187969207764,
+      "learning_rate": 0.00014893333333333334,
+      "loss": 0.5397,
+      "step": 6800
+    },
+    {
+      "epoch": 2.074074074074074,
+      "grad_norm": 0.5677600502967834,
+      "learning_rate": 0.0001444888888888889,
+      "loss": 0.5567,
+      "step": 7000
+    },
+    {
+      "epoch": 2.074074074074074,
+      "eval_loss": 0.49609723687171936,
+      "eval_runtime": 29.4825,
+      "eval_samples_per_second": 101.755,
+      "eval_steps_per_second": 12.719,
+      "step": 7000
+    },
+    {
+      "epoch": 2.1333333333333333,
+      "grad_norm": 0.5369960069656372,
+      "learning_rate": 0.00014004444444444444,
+      "loss": 0.5591,
+      "step": 7200
+    },
+    {
+      "epoch": 2.1925925925925926,
+      "grad_norm": 0.353333055973053,
+      "learning_rate": 0.0001356,
+      "loss": 0.5717,
+      "step": 7400
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "eval_loss": 0.49044421315193176,
+      "eval_runtime": 29.573,
+      "eval_samples_per_second": 101.444,
+      "eval_steps_per_second": 12.68,
+      "step": 7500
+    },
+    {
+      "epoch": 2.251851851851852,
+      "grad_norm": 0.45505258440971375,
+      "learning_rate": 0.00013115555555555554,
+      "loss": 0.5576,
+      "step": 7600
+    },
+    {
+      "epoch": 2.311111111111111,
+      "grad_norm": 0.4602493643760681,
+      "learning_rate": 0.00012671111111111112,
+      "loss": 0.5407,
+      "step": 7800
+    },
+    {
+      "epoch": 2.3703703703703702,
+      "grad_norm": 0.4852631688117981,
+      "learning_rate": 0.00012226666666666667,
+      "loss": 0.5523,
+      "step": 8000
+    },
+    {
+      "epoch": 2.3703703703703702,
+      "eval_loss": 0.48393580317497253,
+      "eval_runtime": 29.3448,
+      "eval_samples_per_second": 102.233,
+      "eval_steps_per_second": 12.779,
+      "step": 8000
+    },
+    {
+      "epoch": 2.4296296296296296,
+      "grad_norm": 0.4038563668727875,
+      "learning_rate": 0.0001178222222222222,
+      "loss": 0.5313,
+      "step": 8200
+    },
+    {
+      "epoch": 2.488888888888889,
+      "grad_norm": 0.5276423692703247,
+      "learning_rate": 0.00011339999999999999,
+      "loss": 0.5489,
+      "step": 8400
+    },
+    {
+      "epoch": 2.5185185185185186,
+      "eval_loss": 0.47875121235847473,
+      "eval_runtime": 29.4918,
+      "eval_samples_per_second": 101.723,
+      "eval_steps_per_second": 12.715,
+      "step": 8500
+    },
+    {
+      "epoch": 2.5481481481481483,
+      "grad_norm": 0.539804995059967,
+      "learning_rate": 0.00010895555555555554,
+      "loss": 0.5456,
+      "step": 8600
+    },
+    {
+      "epoch": 2.6074074074074076,
+      "grad_norm": 0.629550039768219,
+      "learning_rate": 0.0001045111111111111,
+      "loss": 0.5523,
+      "step": 8800
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 0.6091557741165161,
+      "learning_rate": 0.00010006666666666666,
+      "loss": 0.5358,
+      "step": 9000
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "eval_loss": 0.4742780327796936,
+      "eval_runtime": 29.4044,
+      "eval_samples_per_second": 102.026,
+      "eval_steps_per_second": 12.753,
+      "step": 9000
+    },
+    {
+      "epoch": 2.725925925925926,
+      "grad_norm": 0.5220003724098206,
+      "learning_rate": 9.56222222222222e-05,
+      "loss": 0.5398,
+      "step": 9200
+    },
+    {
+      "epoch": 2.785185185185185,
+      "grad_norm": 0.4543071389198303,
+      "learning_rate": 9.117777777777777e-05,
+      "loss": 0.53,
+      "step": 9400
+    },
+    {
+      "epoch": 2.814814814814815,
+      "eval_loss": 0.4698003828525543,
+      "eval_runtime": 29.5022,
+      "eval_samples_per_second": 101.687,
+      "eval_steps_per_second": 12.711,
+      "step": 9500
+    },
+    {
+      "epoch": 2.8444444444444446,
+      "grad_norm": 0.4849318265914917,
+      "learning_rate": 8.673333333333332e-05,
+      "loss": 0.5278,
+      "step": 9600
+    },
+    {
+      "epoch": 2.9037037037037035,
+      "grad_norm": 0.40453964471817017,
+      "learning_rate": 8.228888888888888e-05,
+      "loss": 0.5448,
+      "step": 9800
+    },
+    {
+      "epoch": 2.962962962962963,
+      "grad_norm": 0.5250543355941772,
+      "learning_rate": 7.784444444444443e-05,
+      "loss": 0.538,
+      "step": 10000
+    },
+    {
+      "epoch": 2.962962962962963,
+      "eval_loss": 0.46695125102996826,
+      "eval_runtime": 29.5114,
+      "eval_samples_per_second": 101.656,
+      "eval_steps_per_second": 12.707,
+      "step": 10000
+    },
+    {
+      "epoch": 3.022222222222222,
+      "grad_norm": 0.46158769726753235,
+      "learning_rate": 7.34e-05,
+      "loss": 0.5379,
+      "step": 10200
+    },
+    {
+      "epoch": 3.0814814814814815,
+      "grad_norm": 0.5191985368728638,
+      "learning_rate": 6.895555555555556e-05,
+      "loss": 0.522,
+      "step": 10400
+    },
+    {
+      "epoch": 3.111111111111111,
+      "eval_loss": 0.46375858783721924,
+      "eval_runtime": 29.4058,
+      "eval_samples_per_second": 102.021,
+      "eval_steps_per_second": 12.753,
+      "step": 10500
+    },
+    {
+      "epoch": 3.140740740740741,
+      "grad_norm": 0.5155138969421387,
+      "learning_rate": 6.451111111111111e-05,
+      "loss": 0.5137,
+      "step": 10600
+    },
+    {
+      "epoch": 3.2,
+      "grad_norm": 0.6184846758842468,
+      "learning_rate": 6.006666666666666e-05,
+      "loss": 0.5398,
+      "step": 10800
+    },
+    {
+      "epoch": 3.259259259259259,
+      "grad_norm": 0.48932573199272156,
+      "learning_rate": 5.5622222222222215e-05,
+      "loss": 0.5293,
+      "step": 11000
+    },
+    {
+      "epoch": 3.259259259259259,
+      "eval_loss": 0.4608609676361084,
+      "eval_runtime": 29.8123,
+      "eval_samples_per_second": 100.63,
+      "eval_steps_per_second": 12.579,
+      "step": 11000
+    }
+  ],
+  "logging_steps": 200,
+  "max_steps": 13500,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 4,
+  "save_steps": 1000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7322688552960000.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-11000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5b2e2a527ba622332dbb2398486338b6fc135483c9c1c937ef5cefadf332325
+size 5304

checkpoint-12000/README.md

@@ -0,0 +1,207 @@
+---
+base_model: distilgpt2
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:distilgpt2
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.0

checkpoint-12000/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "distilgpt2",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_proj",
+    "c_attn"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-12000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52b377b2b2769700bb41ee946eb12f0becc601b221b3a839cccaa392495830f6
+size 3248616

checkpoint-12000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf85d0480c0f2c80ca06340328baf17ed7f743a49a085abb27f750563eb4c1d0
+size 6518714