Upload folder using huggingface_hub
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- .gitattributes +4 -0
- codellama-7b-lora-safecoder/checkpoint-last/README.md +202 -0
- codellama-7b-lora-safecoder/checkpoint-last/adapter_config.json +33 -0
- codellama-7b-lora-safecoder/checkpoint-last/adapter_model.safetensors +3 -0
- codellama-7b-lora-safecoder/checkpoint-last/special_tokens_map.json +29 -0
- codellama-7b-lora-safecoder/checkpoint-last/tokenizer.json +0 -0
- codellama-7b-lora-safecoder/checkpoint-last/tokenizer.model +3 -0
- codellama-7b-lora-safecoder/checkpoint-last/tokenizer_config.json +84 -0
- codellama-7b-lora-safecoder/train.log +236 -0
- deepseek-coder-1.3b-lora-safecoder/checkpoint-last/README.md +202 -0
- deepseek-coder-1.3b-lora-safecoder/checkpoint-last/adapter_config.json +33 -0
- deepseek-coder-1.3b-lora-safecoder/checkpoint-last/adapter_model.safetensors +3 -0
- deepseek-coder-1.3b-lora-safecoder/checkpoint-last/special_tokens_map.json +23 -0
- deepseek-coder-1.3b-lora-safecoder/checkpoint-last/tokenizer.json +0 -0
- deepseek-coder-1.3b-lora-safecoder/checkpoint-last/tokenizer_config.json +194 -0
- deepseek-coder-1.3b-lora-safecoder/train.log +93 -0
- deepseek-coder-6.7b-lora-safecoder/checkpoint-last/README.md +202 -0
- deepseek-coder-6.7b-lora-safecoder/checkpoint-last/adapter_config.json +33 -0
- deepseek-coder-6.7b-lora-safecoder/checkpoint-last/adapter_model.safetensors +3 -0
- deepseek-coder-6.7b-lora-safecoder/checkpoint-last/special_tokens_map.json +23 -0
- deepseek-coder-6.7b-lora-safecoder/checkpoint-last/tokenizer.json +0 -0
- deepseek-coder-6.7b-lora-safecoder/checkpoint-last/tokenizer_config.json +194 -0
- deepseek-coder-6.7b-lora-safecoder/train.log +0 -0
- llama2-7b-lora-safecoder/checkpoint-last/README.md +202 -0
- llama2-7b-lora-safecoder/checkpoint-last/adapter_config.json +33 -0
- llama2-7b-lora-safecoder/checkpoint-last/adapter_model.safetensors +3 -0
- llama2-7b-lora-safecoder/checkpoint-last/special_tokens_map.json +23 -0
- llama2-7b-lora-safecoder/checkpoint-last/tokenizer.json +0 -0
- llama2-7b-lora-safecoder/checkpoint-last/tokenizer.model +3 -0
- llama2-7b-lora-safecoder/checkpoint-last/tokenizer_config.json +43 -0
- llama2-7b-lora-safecoder/train.log +64 -0
- mistral-7b-lora-safecoder/checkpoint-last/README.md +202 -0
- mistral-7b-lora-safecoder/checkpoint-last/adapter_config.json +33 -0
- mistral-7b-lora-safecoder/checkpoint-last/adapter_model.safetensors +3 -0
- mistral-7b-lora-safecoder/checkpoint-last/special_tokens_map.json +23 -0
- mistral-7b-lora-safecoder/checkpoint-last/tokenizer.json +0 -0
- mistral-7b-lora-safecoder/checkpoint-last/tokenizer.model +3 -0
- mistral-7b-lora-safecoder/checkpoint-last/tokenizer_config.json +43 -0
- mistral-7b-lora-safecoder/train.log +64 -0
- phi-2-lora-safecoder/checkpoint-last/README.md +202 -0
- phi-2-lora-safecoder/checkpoint-last/adapter_config.json +32 -0
- phi-2-lora-safecoder/checkpoint-last/adapter_model.safetensors +3 -0
- phi-2-lora-safecoder/checkpoint-last/added_tokens.json +40 -0
- phi-2-lora-safecoder/checkpoint-last/merges.txt +0 -0
- phi-2-lora-safecoder/checkpoint-last/special_tokens_map.json +23 -0
- phi-2-lora-safecoder/checkpoint-last/tokenizer.json +0 -0
- phi-2-lora-safecoder/checkpoint-last/tokenizer_config.json +325 -0
- phi-2-lora-safecoder/checkpoint-last/vocab.json +0 -0
- phi-2-lora-safecoder/train.log +65 -0
- qwen2.5-coder-3b-lora-safecoder/checkpoint-last/README.md +202 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
qwen2.5-coder-3b-lora-safecoder/checkpoint-last/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
qwen2.5-coder-7b-lora-safecoder/checkpoint-last/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
qwen3-4b-lora-safecoder/checkpoint-last/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
qwen3-8b-lora-safecoder/checkpoint-last/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
codellama-7b-lora-safecoder/checkpoint-last/README.md
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: codellama/CodeLlama-7b-hf
|
| 3 |
+
library_name: peft
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Model Card for Model ID
|
| 7 |
+
|
| 8 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
## Model Details
|
| 13 |
+
|
| 14 |
+
### Model Description
|
| 15 |
+
|
| 16 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
- **Developed by:** [More Information Needed]
|
| 21 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 22 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 23 |
+
- **Model type:** [More Information Needed]
|
| 24 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 25 |
+
- **License:** [More Information Needed]
|
| 26 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 27 |
+
|
| 28 |
+
### Model Sources [optional]
|
| 29 |
+
|
| 30 |
+
<!-- Provide the basic links for the model. -->
|
| 31 |
+
|
| 32 |
+
- **Repository:** [More Information Needed]
|
| 33 |
+
- **Paper [optional]:** [More Information Needed]
|
| 34 |
+
- **Demo [optional]:** [More Information Needed]
|
| 35 |
+
|
| 36 |
+
## Uses
|
| 37 |
+
|
| 38 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 39 |
+
|
| 40 |
+
### Direct Use
|
| 41 |
+
|
| 42 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 43 |
+
|
| 44 |
+
[More Information Needed]
|
| 45 |
+
|
| 46 |
+
### Downstream Use [optional]
|
| 47 |
+
|
| 48 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 49 |
+
|
| 50 |
+
[More Information Needed]
|
| 51 |
+
|
| 52 |
+
### Out-of-Scope Use
|
| 53 |
+
|
| 54 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 55 |
+
|
| 56 |
+
[More Information Needed]
|
| 57 |
+
|
| 58 |
+
## Bias, Risks, and Limitations
|
| 59 |
+
|
| 60 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 61 |
+
|
| 62 |
+
[More Information Needed]
|
| 63 |
+
|
| 64 |
+
### Recommendations
|
| 65 |
+
|
| 66 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 67 |
+
|
| 68 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 69 |
+
|
| 70 |
+
## How to Get Started with the Model
|
| 71 |
+
|
| 72 |
+
Use the code below to get started with the model.
|
| 73 |
+
|
| 74 |
+
[More Information Needed]
|
| 75 |
+
|
| 76 |
+
## Training Details
|
| 77 |
+
|
| 78 |
+
### Training Data
|
| 79 |
+
|
| 80 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 81 |
+
|
| 82 |
+
[More Information Needed]
|
| 83 |
+
|
| 84 |
+
### Training Procedure
|
| 85 |
+
|
| 86 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 87 |
+
|
| 88 |
+
#### Preprocessing [optional]
|
| 89 |
+
|
| 90 |
+
[More Information Needed]
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
#### Training Hyperparameters
|
| 94 |
+
|
| 95 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 96 |
+
|
| 97 |
+
#### Speeds, Sizes, Times [optional]
|
| 98 |
+
|
| 99 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 100 |
+
|
| 101 |
+
[More Information Needed]
|
| 102 |
+
|
| 103 |
+
## Evaluation
|
| 104 |
+
|
| 105 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 106 |
+
|
| 107 |
+
### Testing Data, Factors & Metrics
|
| 108 |
+
|
| 109 |
+
#### Testing Data
|
| 110 |
+
|
| 111 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 112 |
+
|
| 113 |
+
[More Information Needed]
|
| 114 |
+
|
| 115 |
+
#### Factors
|
| 116 |
+
|
| 117 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 118 |
+
|
| 119 |
+
[More Information Needed]
|
| 120 |
+
|
| 121 |
+
#### Metrics
|
| 122 |
+
|
| 123 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 124 |
+
|
| 125 |
+
[More Information Needed]
|
| 126 |
+
|
| 127 |
+
### Results
|
| 128 |
+
|
| 129 |
+
[More Information Needed]
|
| 130 |
+
|
| 131 |
+
#### Summary
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
## Model Examination [optional]
|
| 136 |
+
|
| 137 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 138 |
+
|
| 139 |
+
[More Information Needed]
|
| 140 |
+
|
| 141 |
+
## Environmental Impact
|
| 142 |
+
|
| 143 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 144 |
+
|
| 145 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 146 |
+
|
| 147 |
+
- **Hardware Type:** [More Information Needed]
|
| 148 |
+
- **Hours used:** [More Information Needed]
|
| 149 |
+
- **Cloud Provider:** [More Information Needed]
|
| 150 |
+
- **Compute Region:** [More Information Needed]
|
| 151 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 152 |
+
|
| 153 |
+
## Technical Specifications [optional]
|
| 154 |
+
|
| 155 |
+
### Model Architecture and Objective
|
| 156 |
+
|
| 157 |
+
[More Information Needed]
|
| 158 |
+
|
| 159 |
+
### Compute Infrastructure
|
| 160 |
+
|
| 161 |
+
[More Information Needed]
|
| 162 |
+
|
| 163 |
+
#### Hardware
|
| 164 |
+
|
| 165 |
+
[More Information Needed]
|
| 166 |
+
|
| 167 |
+
#### Software
|
| 168 |
+
|
| 169 |
+
[More Information Needed]
|
| 170 |
+
|
| 171 |
+
## Citation [optional]
|
| 172 |
+
|
| 173 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 174 |
+
|
| 175 |
+
**BibTeX:**
|
| 176 |
+
|
| 177 |
+
[More Information Needed]
|
| 178 |
+
|
| 179 |
+
**APA:**
|
| 180 |
+
|
| 181 |
+
[More Information Needed]
|
| 182 |
+
|
| 183 |
+
## Glossary [optional]
|
| 184 |
+
|
| 185 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 186 |
+
|
| 187 |
+
[More Information Needed]
|
| 188 |
+
|
| 189 |
+
## More Information [optional]
|
| 190 |
+
|
| 191 |
+
[More Information Needed]
|
| 192 |
+
|
| 193 |
+
## Model Card Authors [optional]
|
| 194 |
+
|
| 195 |
+
[More Information Needed]
|
| 196 |
+
|
| 197 |
+
## Model Card Contact
|
| 198 |
+
|
| 199 |
+
[More Information Needed]
|
| 200 |
+
### Framework versions
|
| 201 |
+
|
| 202 |
+
- PEFT 0.8.2
|
codellama-7b-lora-safecoder/checkpoint-last/adapter_config.json
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alpha_pattern": {},
|
| 3 |
+
"auto_mapping": null,
|
| 4 |
+
"base_model_name_or_path": "codellama/CodeLlama-7b-hf",
|
| 5 |
+
"bias": "none",
|
| 6 |
+
"fan_in_fan_out": false,
|
| 7 |
+
"inference_mode": true,
|
| 8 |
+
"init_lora_weights": true,
|
| 9 |
+
"layers_pattern": null,
|
| 10 |
+
"layers_to_transform": null,
|
| 11 |
+
"loftq_config": {},
|
| 12 |
+
"lora_alpha": 32,
|
| 13 |
+
"lora_dropout": 0.1,
|
| 14 |
+
"megatron_config": null,
|
| 15 |
+
"megatron_core": "megatron.core",
|
| 16 |
+
"modules_to_save": null,
|
| 17 |
+
"peft_type": "LORA",
|
| 18 |
+
"r": 16,
|
| 19 |
+
"rank_pattern": {},
|
| 20 |
+
"revision": null,
|
| 21 |
+
"target_modules": [
|
| 22 |
+
"q_proj",
|
| 23 |
+
"gate_proj",
|
| 24 |
+
"k_proj",
|
| 25 |
+
"v_proj",
|
| 26 |
+
"up_proj",
|
| 27 |
+
"o_proj",
|
| 28 |
+
"down_proj",
|
| 29 |
+
"lm_head"
|
| 30 |
+
],
|
| 31 |
+
"task_type": "CAUSAL_LM",
|
| 32 |
+
"use_rslora": false
|
| 33 |
+
}
|
codellama-7b-lora-safecoder/checkpoint-last/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fc1f6b20cc17e8ef765d5f49102a05e3d8f2198feb4edb9b7bd74c5fe5ea6ec5
|
| 3 |
+
size 686830160
|
codellama-7b-lora-safecoder/checkpoint-last/special_tokens_map.json
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"additional_special_tokens": [
|
| 3 |
+
"▁<PRE>",
|
| 4 |
+
"▁<MID>",
|
| 5 |
+
"▁<SUF>",
|
| 6 |
+
"▁<EOT>"
|
| 7 |
+
],
|
| 8 |
+
"bos_token": {
|
| 9 |
+
"content": "<s>",
|
| 10 |
+
"lstrip": false,
|
| 11 |
+
"normalized": false,
|
| 12 |
+
"rstrip": false,
|
| 13 |
+
"single_word": false
|
| 14 |
+
},
|
| 15 |
+
"eos_token": {
|
| 16 |
+
"content": "</s>",
|
| 17 |
+
"lstrip": false,
|
| 18 |
+
"normalized": false,
|
| 19 |
+
"rstrip": false,
|
| 20 |
+
"single_word": false
|
| 21 |
+
},
|
| 22 |
+
"unk_token": {
|
| 23 |
+
"content": "<unk>",
|
| 24 |
+
"lstrip": false,
|
| 25 |
+
"normalized": false,
|
| 26 |
+
"rstrip": false,
|
| 27 |
+
"single_word": false
|
| 28 |
+
}
|
| 29 |
+
}
|
codellama-7b-lora-safecoder/checkpoint-last/tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
codellama-7b-lora-safecoder/checkpoint-last/tokenizer.model
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:45ccb9c8b6b561889acea59191d66986d314e7cbd6a78abc6e49b139ca91c1e6
|
| 3 |
+
size 500058
|
codellama-7b-lora-safecoder/checkpoint-last/tokenizer_config.json
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_bos_token": true,
|
| 3 |
+
"add_eos_token": false,
|
| 4 |
+
"added_tokens_decoder": {
|
| 5 |
+
"0": {
|
| 6 |
+
"content": "<unk>",
|
| 7 |
+
"lstrip": false,
|
| 8 |
+
"normalized": false,
|
| 9 |
+
"rstrip": false,
|
| 10 |
+
"single_word": false,
|
| 11 |
+
"special": true
|
| 12 |
+
},
|
| 13 |
+
"1": {
|
| 14 |
+
"content": "<s>",
|
| 15 |
+
"lstrip": false,
|
| 16 |
+
"normalized": false,
|
| 17 |
+
"rstrip": false,
|
| 18 |
+
"single_word": false,
|
| 19 |
+
"special": true
|
| 20 |
+
},
|
| 21 |
+
"2": {
|
| 22 |
+
"content": "</s>",
|
| 23 |
+
"lstrip": false,
|
| 24 |
+
"normalized": false,
|
| 25 |
+
"rstrip": false,
|
| 26 |
+
"single_word": false,
|
| 27 |
+
"special": true
|
| 28 |
+
},
|
| 29 |
+
"32007": {
|
| 30 |
+
"content": "▁<PRE>",
|
| 31 |
+
"lstrip": false,
|
| 32 |
+
"normalized": false,
|
| 33 |
+
"rstrip": false,
|
| 34 |
+
"single_word": false,
|
| 35 |
+
"special": true
|
| 36 |
+
},
|
| 37 |
+
"32008": {
|
| 38 |
+
"content": "▁<SUF>",
|
| 39 |
+
"lstrip": false,
|
| 40 |
+
"normalized": false,
|
| 41 |
+
"rstrip": false,
|
| 42 |
+
"single_word": false,
|
| 43 |
+
"special": true
|
| 44 |
+
},
|
| 45 |
+
"32009": {
|
| 46 |
+
"content": "▁<MID>",
|
| 47 |
+
"lstrip": false,
|
| 48 |
+
"normalized": false,
|
| 49 |
+
"rstrip": false,
|
| 50 |
+
"single_word": false,
|
| 51 |
+
"special": true
|
| 52 |
+
},
|
| 53 |
+
"32010": {
|
| 54 |
+
"content": "▁<EOT>",
|
| 55 |
+
"lstrip": false,
|
| 56 |
+
"normalized": false,
|
| 57 |
+
"rstrip": false,
|
| 58 |
+
"single_word": false,
|
| 59 |
+
"special": true
|
| 60 |
+
}
|
| 61 |
+
},
|
| 62 |
+
"additional_special_tokens": [
|
| 63 |
+
"▁<PRE>",
|
| 64 |
+
"▁<MID>",
|
| 65 |
+
"▁<SUF>",
|
| 66 |
+
"▁<EOT>"
|
| 67 |
+
],
|
| 68 |
+
"bos_token": "<s>",
|
| 69 |
+
"clean_up_tokenization_spaces": false,
|
| 70 |
+
"eos_token": "</s>",
|
| 71 |
+
"eot_token": "▁<EOT>",
|
| 72 |
+
"extra_special_tokens": {},
|
| 73 |
+
"fill_token": "<FILL_ME>",
|
| 74 |
+
"legacy": null,
|
| 75 |
+
"middle_token": "▁<MID>",
|
| 76 |
+
"model_max_length": 1000000000000000019884624838656,
|
| 77 |
+
"pad_token": null,
|
| 78 |
+
"prefix_token": "▁<PRE>",
|
| 79 |
+
"sp_model_kwargs": {},
|
| 80 |
+
"suffix_token": "▁<SUF>",
|
| 81 |
+
"tokenizer_class": "CodeLlamaTokenizer",
|
| 82 |
+
"unk_token": "<unk>",
|
| 83 |
+
"use_default_system_prompt": false
|
| 84 |
+
}
|
codellama-7b-lora-safecoder/train.log
ADDED
|
@@ -0,0 +1,236 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
05/12/2026 04:15:00 - INFO - accelerate.utils.modeling - We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can set `max_memory` in to a higher value to use more memory (at your own risk).
|
| 2 |
+
05/12/2026 04:15:34 - INFO - root - number of sec samples before upsampling: 1806
|
| 3 |
+
05/12/2026 04:15:34 - INFO - root - number of sec samples after upsampling: 2394
|
| 4 |
+
05/12/2026 04:15:37 - INFO - root - Training args Namespace(output_name='codellama-7b-lora-safecoder', datasets=['evol', 'sec-desc', 'sec-new-desc'], pretrain_name='codellama-7b', loss_weight=1.0, sven=False, num_train_epochs=5, learning_rate=0.001, max_num_tokens=1024, batch_size=1, grad_acc_steps=16, weight_decay=0.01, adam_epsilon=1e-08, warmup_steps=0, max_grad_norm=1.0, dropout=0.1, kl_loss_weight=0, exclude_neg=False, no_weights=False, lora=True, r=16, lora_alpha=32, lora_dropout=0.1, sampling_size=20, sampling_method='minority', cwes=['all'], langs=['all'], logging_steps=50, save_epochs=10, seed=2, data_dir='../data_train_val', model_dir='../trained/', output_dir='../trained/codellama-7b-lora-safecoder', logger=<RootLogger root (INFO)>)
|
| 5 |
+
05/12/2026 04:15:37 - INFO - root - ***** Running training *****
|
| 6 |
+
05/12/2026 04:15:37 - INFO - root - Num samples = 30692
|
| 7 |
+
05/12/2026 04:15:37 - INFO - root - Num epoch = 5
|
| 8 |
+
05/12/2026 04:15:37 - INFO - root - Batch size= 1
|
| 9 |
+
05/12/2026 04:15:37 - INFO - root - Total batch size (w. accumulation) = 16
|
| 10 |
+
05/12/2026 04:15:37 - INFO - root - Gradient Accumulation steps = 16
|
| 11 |
+
05/12/2026 04:15:37 - INFO - root - Total optimization steps = 9590
|
| 12 |
+
05/12/2026 04:15:37 - INFO - root - Num val samples = 3339
|
| 13 |
+
05/12/2026 04:15:37 - INFO - root - Num parameters = 6779101440
|
| 14 |
+
05/12/2026 04:15:37 - INFO - root - Num trainable parameters = 40554752
|
| 15 |
+
05/12/2026 04:21:02 - INFO - root - epochs: 1/5, steps: 50/9590, func: 0.050948, pos: 0.066863, neg: 0.102675, 0%: 17h 13m 50s
|
| 16 |
+
05/12/2026 04:26:32 - INFO - root - epochs: 1/5, steps: 100/9590, func: 0.047577, pos: 0.074183, neg: 0.041983, 1%: 17h 15m 36s
|
| 17 |
+
05/12/2026 04:31:56 - INFO - root - epochs: 1/5, steps: 150/9590, func: 0.048479, pos: 0.083375, neg: 0.044575, 1%: 17h 7m 9s
|
| 18 |
+
05/12/2026 04:37:28 - INFO - root - epochs: 1/5, steps: 200/9590, func: 0.049801, pos: 0.100972, neg: 0.029368, 2%: 17h 5m 57s
|
| 19 |
+
05/12/2026 04:42:57 - INFO - root - epochs: 1/5, steps: 250/9590, func: 0.048792, pos: 0.109971, neg: 0.047552, 2%: 17h 1m 3s
|
| 20 |
+
05/12/2026 04:48:33 - INFO - root - epochs: 1/5, steps: 300/9590, func: 0.050624, pos: 0.096543, neg: 0.028878, 3%: 17h 0m 26s
|
| 21 |
+
05/12/2026 04:53:57 - INFO - root - epochs: 1/5, steps: 350/9590, func: 0.049892, pos: 0.131824, neg: 0.033636, 3%: 16h 52m 16s
|
| 22 |
+
05/12/2026 04:59:21 - INFO - root - epochs: 1/5, steps: 400/9590, func: 0.051175, pos: 0.098686, neg: 0.026805, 4%: 16h 45m 38s
|
| 23 |
+
05/12/2026 05:04:53 - INFO - root - epochs: 1/5, steps: 450/9590, func: 0.050767, pos: 0.132716, neg: 0.029258, 4%: 16h 40m 58s
|
| 24 |
+
05/12/2026 05:10:22 - INFO - root - epochs: 1/5, steps: 500/9590, func: 0.051894, pos: 0.139853, neg: 0.040111, 5%: 16h 35m 38s
|
| 25 |
+
05/12/2026 05:15:46 - INFO - root - epochs: 1/5, steps: 550/9590, func: 0.050903, pos: 0.099576, neg: 0.036543, 5%: 16h 28m 50s
|
| 26 |
+
05/12/2026 05:21:17 - INFO - root - epochs: 1/5, steps: 600/9590, func: 0.050571, pos: 0.105049, neg: 0.035279, 6%: 16h 24m 7s
|
| 27 |
+
05/12/2026 05:26:49 - INFO - root - epochs: 1/5, steps: 650/9590, func: 0.051513, pos: 0.08535, neg: 0.026287, 6%: 16h 19m 19s
|
| 28 |
+
05/12/2026 05:32:16 - INFO - root - epochs: 1/5, steps: 700/9590, func: 0.051382, pos: 0.11401, neg: 0.022054, 7%: 16h 13m 37s
|
| 29 |
+
05/12/2026 05:37:42 - INFO - root - epochs: 1/5, steps: 750/9590, func: 0.052051, pos: 0.110181, neg: 0.022295, 7%: 16h 7m 50s
|
| 30 |
+
05/12/2026 05:41:40 - INFO - accelerate.utils.modeling - We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can set `max_memory` in to a higher value to use more memory (at your own risk).
|
| 31 |
+
05/12/2026 05:42:09 - INFO - root - number of sec samples before upsampling: 1806
|
| 32 |
+
05/12/2026 05:42:09 - INFO - root - number of sec samples after upsampling: 2394
|
| 33 |
+
05/12/2026 05:42:12 - INFO - root - Training args Namespace(output_name='codellama-7b-lora-safecoder', datasets=['evol', 'sec-desc', 'sec-new-desc'], pretrain_name='codellama-7b', loss_weight=1.0, sven=False, num_train_epochs=5, learning_rate=0.001, max_num_tokens=1024, batch_size=1, grad_acc_steps=16, weight_decay=0.01, adam_epsilon=1e-08, warmup_steps=0, max_grad_norm=1.0, dropout=0.1, kl_loss_weight=0, exclude_neg=False, no_weights=False, lora=True, r=16, lora_alpha=32, lora_dropout=0.1, sampling_size=20, sampling_method='minority', cwes=['all'], langs=['all'], logging_steps=50, save_epochs=10, seed=2, data_dir='../data_train_val', model_dir='../trained/', output_dir='../trained/codellama-7b-lora-safecoder', logger=<RootLogger root (INFO)>)
|
| 34 |
+
05/12/2026 05:42:12 - INFO - root - ***** Running training *****
|
| 35 |
+
05/12/2026 05:42:12 - INFO - root - Num samples = 30692
|
| 36 |
+
05/12/2026 05:42:12 - INFO - root - Num epoch = 5
|
| 37 |
+
05/12/2026 05:42:12 - INFO - root - Batch size= 1
|
| 38 |
+
05/12/2026 05:42:12 - INFO - root - Total batch size (w. accumulation) = 16
|
| 39 |
+
05/12/2026 05:42:12 - INFO - root - Gradient Accumulation steps = 16
|
| 40 |
+
05/12/2026 05:42:12 - INFO - root - Total optimization steps = 9590
|
| 41 |
+
05/12/2026 05:42:12 - INFO - root - Num val samples = 3339
|
| 42 |
+
05/12/2026 05:42:12 - INFO - root - Num parameters = 6779101440
|
| 43 |
+
05/12/2026 05:42:12 - INFO - root - Num trainable parameters = 40554752
|
| 44 |
+
05/12/2026 05:46:16 - INFO - root - epochs: 1/5, steps: 50/9590, func: 0.05094, pos: 0.066244, neg: 0.104063, 0%: 12h 56m 45s
|
| 45 |
+
05/12/2026 05:50:23 - INFO - root - epochs: 1/5, steps: 100/9590, func: 0.049307, pos: 0.077082, neg: 0.04401, 1%: 12h 55m 55s
|
| 46 |
+
05/12/2026 05:54:25 - INFO - root - epochs: 1/5, steps: 150/9590, func: 0.051967, pos: 0.102716, neg: 0.041288, 1%: 12h 48m 36s
|
| 47 |
+
05/12/2026 05:58:31 - INFO - root - epochs: 1/5, steps: 200/9590, func: 0.04968, pos: 0.089183, neg: 0.03749, 2%: 12h 45m 58s
|
| 48 |
+
05/12/2026 06:02:37 - INFO - root - epochs: 1/5, steps: 250/9590, func: 0.048648, pos: 0.089534, neg: 0.056231, 2%: 12h 42m 50s
|
| 49 |
+
05/12/2026 06:06:49 - INFO - root - epochs: 1/5, steps: 300/9590, func: 0.050689, pos: 0.087186, neg: 0.035415, 3%: 12h 42m 28s
|
| 50 |
+
05/12/2026 06:10:50 - INFO - root - epochs: 1/5, steps: 350/9590, func: 0.050049, pos: 0.11521, neg: 0.036819, 3%: 12h 36m 12s
|
| 51 |
+
05/12/2026 06:14:51 - INFO - root - epochs: 1/5, steps: 400/9590, func: 0.051514, pos: 0.094837, neg: 0.029605, 4%: 12h 30m 41s
|
| 52 |
+
05/12/2026 06:18:58 - INFO - root - epochs: 1/5, steps: 450/9590, func: 0.050486, pos: 0.142797, neg: 0.023274, 4%: 12h 27m 11s
|
| 53 |
+
05/12/2026 06:23:02 - INFO - root - epochs: 1/5, steps: 500/9590, func: 0.052427, pos: 0.131603, neg: 0.040929, 5%: 12h 22m 24s
|
| 54 |
+
05/12/2026 06:27:00 - INFO - root - epochs: 1/5, steps: 550/9590, func: 0.051054, pos: 0.111602, neg: 0.036933, 5%: 12h 16m 23s
|
| 55 |
+
05/12/2026 06:31:03 - INFO - root - epochs: 1/5, steps: 600/9590, func: 0.050244, pos: 0.093686, neg: 0.032851, 6%: 12h 11m 56s
|
| 56 |
+
05/12/2026 06:35:05 - INFO - root - epochs: 1/5, steps: 650/9590, func: 0.051575, pos: 0.093165, neg: 0.028331, 6%: 12h 7m 26s
|
| 57 |
+
05/12/2026 06:39:06 - INFO - root - epochs: 1/5, steps: 700/9590, func: 0.051052, pos: 0.127034, neg: 0.017673, 7%: 12h 2m 36s
|
| 58 |
+
05/12/2026 06:43:09 - INFO - root - epochs: 1/5, steps: 750/9590, func: 0.051672, pos: 0.106718, neg: 0.021583, 7%: 11h 58m 27s
|
| 59 |
+
05/12/2026 06:47:11 - INFO - root - epochs: 1/5, steps: 800/9590, func: 0.052223, pos: 0.199383, neg: 0.018326, 8%: 11h 54m 17s
|
| 60 |
+
05/12/2026 06:51:14 - INFO - root - epochs: 1/5, steps: 850/9590, func: 0.051818, pos: 0.131503, neg: 0.017953, 8%: 11h 49m 55s
|
| 61 |
+
05/12/2026 06:55:19 - INFO - root - epochs: 1/5, steps: 900/9590, func: 0.051805, pos: 0.095854, neg: 0.031777, 9%: 11h 46m 2s
|
| 62 |
+
05/12/2026 06:59:20 - INFO - root - epochs: 1/5, steps: 950/9590, func: 0.051391, pos: 0.114702, neg: 0.024798, 9%: 11h 41m 38s
|
| 63 |
+
05/12/2026 07:03:21 - INFO - root - epochs: 1/5, steps: 1000/9590, func: 0.052208, pos: 0.105047, neg: 0.025137, 10%: 11h 37m 13s
|
| 64 |
+
05/12/2026 07:07:24 - INFO - root - epochs: 1/5, steps: 1050/9590, func: 0.052544, pos: 0.109283, neg: 0.023481, 10%: 11h 33m 8s
|
| 65 |
+
05/12/2026 07:11:27 - INFO - root - epochs: 1/5, steps: 1100/9590, func: 0.052499, pos: 0.074911, neg: 0.017531, 11%: 11h 28m 54s
|
| 66 |
+
05/12/2026 07:15:30 - INFO - root - epochs: 1/5, steps: 1150/9590, func: 0.052749, pos: 0.074735, neg: 0.034612, 11%: 11h 24m 47s
|
| 67 |
+
05/12/2026 07:19:29 - INFO - root - epochs: 1/5, steps: 1200/9590, func: 0.05158, pos: 0.08104, neg: 0.030226, 12%: 11h 20m 21s
|
| 68 |
+
05/12/2026 07:23:31 - INFO - root - epochs: 1/5, steps: 1250/9590, func: 0.05273, pos: 0.112143, neg: 0.023065, 13%: 11h 16m 6s
|
| 69 |
+
05/12/2026 07:27:33 - INFO - root - epochs: 1/5, steps: 1300/9590, func: 0.051978, pos: 0.0801, neg: 0.018668, 13%: 11h 11m 56s
|
| 70 |
+
05/12/2026 07:31:36 - INFO - root - epochs: 1/5, steps: 1350/9590, func: 0.053459, pos: 0.109475, neg: 0.021353, 14%: 11h 7m 53s
|
| 71 |
+
05/12/2026 07:35:37 - INFO - root - epochs: 1/5, steps: 1400/9590, func: 0.051982, pos: 0.099747, neg: 0.017942, 14%: 11h 3m 32s
|
| 72 |
+
05/12/2026 07:39:45 - INFO - root - epochs: 1/5, steps: 1450/9590, func: 0.052103, pos: 0.0787, neg: 0.018889, 15%: 10h 59m 57s
|
| 73 |
+
05/12/2026 07:43:53 - INFO - root - epochs: 1/5, steps: 1500/9590, func: 0.053136, pos: 0.086926, neg: 0.022841, 15%: 10h 56m 18s
|
| 74 |
+
05/12/2026 07:47:58 - INFO - root - epochs: 1/5, steps: 1550/9590, func: 0.051407, pos: 0.074462, neg: 0.033253, 16%: 10h 52m 29s
|
| 75 |
+
05/12/2026 07:52:02 - INFO - root - epochs: 1/5, steps: 1600/9590, func: 0.051884, pos: 0.113117, neg: 0.013997, 16%: 10h 48m 21s
|
| 76 |
+
05/12/2026 07:56:05 - INFO - root - epochs: 1/5, steps: 1650/9590, func: 0.051529, pos: 0.096809, neg: 0.034268, 17%: 10h 44m 22s
|
| 77 |
+
05/12/2026 08:00:07 - INFO - root - epochs: 1/5, steps: 1700/9590, func: 0.053094, pos: 0.094909, neg: 0.025515, 17%: 10h 40m 11s
|
| 78 |
+
05/12/2026 08:04:13 - INFO - root - epochs: 1/5, steps: 1750/9590, func: 0.051572, pos: 0.070656, neg: 0.018058, 18%: 10h 36m 16s
|
| 79 |
+
05/12/2026 08:08:14 - INFO - root - epochs: 1/5, steps: 1800/9590, func: 0.053385, pos: 0.081258, neg: 0.021151, 18%: 10h 32m 3s
|
| 80 |
+
05/12/2026 08:12:15 - INFO - root - epochs: 1/5, steps: 1850/9590, func: 0.051872, pos: 0.082147, neg: 0.025378, 19%: 10h 27m 52s
|
| 81 |
+
05/12/2026 08:16:14 - INFO - root - epochs: 1/5, steps: 1900/9590, func: 0.051771, pos: 0.069222, neg: 0.02802, 19%: 10h 23m 33s
|
| 82 |
+
05/12/2026 08:20:22 - INFO - root - epochs: 2/5, steps: 1950/9590, func: 0.049655, pos: 0.091716, neg: 0.017551, 20%: 10h 19m 51s
|
| 83 |
+
05/12/2026 08:24:27 - INFO - root - epochs: 2/5, steps: 2000/9590, func: 0.048194, pos: 0.089666, neg: 0.016199, 20%: 10h 15m 51s
|
| 84 |
+
05/12/2026 08:28:31 - INFO - root - epochs: 2/5, steps: 2050/9590, func: 0.047949, pos: 0.064842, neg: 0.016331, 21%: 10h 11m 50s
|
| 85 |
+
05/12/2026 08:32:25 - INFO - root - epochs: 2/5, steps: 2100/9590, func: 0.047398, pos: 0.05705, neg: 0.022757, 21%: 10h 7m 14s
|
| 86 |
+
05/12/2026 08:36:23 - INFO - root - epochs: 2/5, steps: 2150/9590, func: 0.046701, pos: 0.047835, neg: 0.007168, 22%: 10h 2m 49s
|
| 87 |
+
05/12/2026 08:40:20 - INFO - root - epochs: 2/5, steps: 2200/9590, func: 0.047965, pos: 0.075317, neg: 0.016316, 22%: 9h 58m 28s
|
| 88 |
+
05/12/2026 08:44:15 - INFO - root - epochs: 2/5, steps: 2250/9590, func: 0.049061, pos: 0.063372, neg: 0.013699, 23%: 9h 53m 59s
|
| 89 |
+
05/12/2026 08:48:25 - INFO - root - epochs: 2/5, steps: 2300/9590, func: 0.049057, pos: 0.048502, neg: 0.018374, 23%: 9h 50m 15s
|
| 90 |
+
05/12/2026 08:52:32 - INFO - root - epochs: 2/5, steps: 2350/9590, func: 0.048354, pos: 0.069101, neg: 0.026868, 24%: 9h 46m 26s
|
| 91 |
+
05/12/2026 08:56:38 - INFO - root - epochs: 2/5, steps: 2400/9590, func: 0.049917, pos: 0.054647, neg: 0.015619, 25%: 9h 42m 34s
|
| 92 |
+
05/12/2026 09:00:41 - INFO - root - epochs: 2/5, steps: 2450/9590, func: 0.049093, pos: 0.066494, neg: 0.018207, 25%: 9h 38m 31s
|
| 93 |
+
05/12/2026 09:04:46 - INFO - root - epochs: 2/5, steps: 2500/9590, func: 0.048869, pos: 0.050998, neg: 0.01707, 26%: 9h 34m 33s
|
| 94 |
+
05/12/2026 09:08:49 - INFO - root - epochs: 2/5, steps: 2550/9590, func: 0.049238, pos: 0.049718, neg: 0.021854, 26%: 9h 30m 32s
|
| 95 |
+
05/12/2026 09:12:54 - INFO - root - epochs: 2/5, steps: 2600/9590, func: 0.048473, pos: 0.048989, neg: 0.010893, 27%: 9h 26m 33s
|
| 96 |
+
05/12/2026 09:16:56 - INFO - root - epochs: 2/5, steps: 2650/9590, func: 0.047705, pos: 0.057008, neg: 0.013472, 27%: 9h 22m 26s
|
| 97 |
+
05/12/2026 09:20:51 - INFO - root - epochs: 2/5, steps: 2700/9590, func: 0.048697, pos: 0.051007, neg: 0.020675, 28%: 9h 18m 3s
|
| 98 |
+
05/12/2026 09:24:59 - INFO - root - epochs: 2/5, steps: 2750/9590, func: 0.049822, pos: 0.066192, neg: 0.019821, 28%: 9h 14m 14s
|
| 99 |
+
05/12/2026 09:29:03 - INFO - root - epochs: 2/5, steps: 2800/9590, func: 0.048675, pos: 0.053753, neg: 0.013854, 29%: 9h 10m 12s
|
| 100 |
+
05/12/2026 09:33:07 - INFO - root - epochs: 2/5, steps: 2850/9590, func: 0.049093, pos: 0.069586, neg: 0.016566, 29%: 9h 6m 10s
|
| 101 |
+
05/12/2026 09:37:04 - INFO - root - epochs: 2/5, steps: 2900/9590, func: 0.048524, pos: 0.070572, neg: 0.017197, 30%: 9h 1m 56s
|
| 102 |
+
05/12/2026 09:41:08 - INFO - root - epochs: 2/5, steps: 2950/9590, func: 0.049634, pos: 0.074833, neg: 0.014874, 30%: 8h 57m 53s
|
| 103 |
+
05/12/2026 09:45:11 - INFO - root - epochs: 2/5, steps: 3000/9590, func: 0.050054, pos: 0.087303, neg: 0.010392, 31%: 8h 53m 50s
|
| 104 |
+
05/12/2026 09:49:13 - INFO - root - epochs: 2/5, steps: 3050/9590, func: 0.050311, pos: 0.062351, neg: 0.018711, 31%: 8h 49m 46s
|
| 105 |
+
05/12/2026 09:53:16 - INFO - root - epochs: 2/5, steps: 3100/9590, func: 0.04941, pos: 0.055028, neg: 0.014512, 32%: 8h 45m 44s
|
| 106 |
+
05/12/2026 09:57:21 - INFO - root - epochs: 2/5, steps: 3150/9590, func: 0.048447, pos: 0.049718, neg: 0.010809, 32%: 8h 41m 45s
|
| 107 |
+
05/12/2026 10:01:26 - INFO - root - epochs: 2/5, steps: 3200/9590, func: 0.048828, pos: 0.04025, neg: 0.011925, 33%: 8h 37m 45s
|
| 108 |
+
05/12/2026 10:05:28 - INFO - root - epochs: 2/5, steps: 3250/9590, func: 0.049547, pos: 0.047331, neg: 0.016769, 33%: 8h 33m 40s
|
| 109 |
+
05/12/2026 10:09:32 - INFO - root - epochs: 2/5, steps: 3300/9590, func: 0.049657, pos: 0.052612, neg: 0.01622, 34%: 8h 29m 38s
|
| 110 |
+
05/12/2026 10:13:36 - INFO - root - epochs: 2/5, steps: 3350/9590, func: 0.048103, pos: 0.051177, neg: 0.016804, 34%: 8h 25m 37s
|
| 111 |
+
05/12/2026 10:17:43 - INFO - root - epochs: 2/5, steps: 3400/9590, func: 0.049741, pos: 0.046914, neg: 0.00997, 35%: 8h 21m 39s
|
| 112 |
+
05/12/2026 10:21:39 - INFO - root - epochs: 2/5, steps: 3450/9590, func: 0.049321, pos: 0.044237, neg: 0.021636, 35%: 8h 17m 24s
|
| 113 |
+
05/12/2026 10:25:41 - INFO - root - epochs: 2/5, steps: 3500/9590, func: 0.048902, pos: 0.054882, neg: 0.014377, 36%: 8h 13m 20s
|
| 114 |
+
05/12/2026 10:29:46 - INFO - root - epochs: 2/5, steps: 3550/9590, func: 0.048824, pos: 0.063807, neg: 0.014463, 37%: 8h 9m 20s
|
| 115 |
+
05/12/2026 10:33:45 - INFO - root - epochs: 2/5, steps: 3600/9590, func: 0.048231, pos: 0.058499, neg: 0.024831, 37%: 8h 5m 11s
|
| 116 |
+
05/12/2026 10:37:44 - INFO - root - epochs: 2/5, steps: 3650/9590, func: 0.048081, pos: 0.085183, neg: 0.014266, 38%: 8h 1m 1s
|
| 117 |
+
05/12/2026 10:41:45 - INFO - root - epochs: 2/5, steps: 3700/9590, func: 0.049161, pos: 0.052952, neg: 0.017932, 38%: 7h 56m 57s
|
| 118 |
+
05/12/2026 10:45:46 - INFO - root - epochs: 2/5, steps: 3750/9590, func: 0.0463, pos: 0.038567, neg: 0.024301, 39%: 7h 52m 49s
|
| 119 |
+
05/12/2026 10:49:48 - INFO - root - epochs: 2/5, steps: 3800/9590, func: 0.049297, pos: 0.069484, neg: 0.014716, 39%: 7h 48m 46s
|
| 120 |
+
05/12/2026 10:53:53 - INFO - root - epochs: 3/5, steps: 3850/9590, func: 0.046816, pos: 0.040204, neg: 0.01251, 40%: 7h 44m 46s
|
| 121 |
+
05/12/2026 10:58:02 - INFO - root - epochs: 3/5, steps: 3900/9590, func: 0.043129, pos: 0.026737, neg: 0.005862, 40%: 7h 40m 51s
|
| 122 |
+
05/12/2026 11:02:08 - INFO - root - epochs: 3/5, steps: 3950/9590, func: 0.042884, pos: 0.033689, neg: 0.018398, 41%: 7h 36m 55s
|
| 123 |
+
05/12/2026 11:06:16 - INFO - root - epochs: 3/5, steps: 4000/9590, func: 0.04335, pos: 0.033484, neg: 0.011237, 41%: 7h 32m 57s
|
| 124 |
+
05/12/2026 11:10:22 - INFO - root - epochs: 3/5, steps: 4050/9590, func: 0.043222, pos: 0.03106, neg: 0.009611, 42%: 7h 28m 58s
|
| 125 |
+
05/12/2026 11:14:26 - INFO - root - epochs: 3/5, steps: 4100/9590, func: 0.04263, pos: 0.024914, neg: 0.0074, 42%: 7h 24m 56s
|
| 126 |
+
05/12/2026 11:18:25 - INFO - root - epochs: 3/5, steps: 4150/9590, func: 0.042534, pos: 0.029838, neg: 0.00935, 43%: 7h 20m 48s
|
| 127 |
+
05/12/2026 11:22:26 - INFO - root - epochs: 3/5, steps: 4200/9590, func: 0.043006, pos: 0.023594, neg: 0.003426, 43%: 7h 16m 44s
|
| 128 |
+
05/12/2026 11:26:27 - INFO - root - epochs: 3/5, steps: 4250/9590, func: 0.043293, pos: 0.038818, neg: 0.0156, 44%: 7h 12m 37s
|
| 129 |
+
05/12/2026 11:30:28 - INFO - root - epochs: 3/5, steps: 4300/9590, func: 0.043924, pos: 0.026801, neg: 0.013046, 44%: 7h 8m 32s
|
| 130 |
+
05/12/2026 11:34:30 - INFO - root - epochs: 3/5, steps: 4350/9590, func: 0.043028, pos: 0.036002, neg: 0.007841, 45%: 7h 4m 27s
|
| 131 |
+
05/12/2026 11:38:26 - INFO - root - epochs: 3/5, steps: 4400/9590, func: 0.04386, pos: 0.032908, neg: 0.013913, 45%: 7h 0m 15s
|
| 132 |
+
05/12/2026 11:42:29 - INFO - root - epochs: 3/5, steps: 4450/9590, func: 0.044371, pos: 0.031435, neg: 0.011, 46%: 6h 56m 13s
|
| 133 |
+
05/12/2026 11:46:32 - INFO - root - epochs: 3/5, steps: 4500/9590, func: 0.043781, pos: 0.026892, neg: 0.004409, 46%: 6h 52m 11s
|
| 134 |
+
05/12/2026 11:50:35 - INFO - root - epochs: 3/5, steps: 4550/9590, func: 0.044126, pos: 0.03361, neg: 0.01439, 47%: 6h 48m 9s
|
| 135 |
+
05/12/2026 11:54:34 - INFO - root - epochs: 3/5, steps: 4600/9590, func: 0.043945, pos: 0.042274, neg: 0.011236, 47%: 6h 44m 0s
|
| 136 |
+
05/12/2026 11:58:32 - INFO - root - epochs: 3/5, steps: 4650/9590, func: 0.043599, pos: 0.020995, neg: 0.012674, 48%: 6h 39m 53s
|
| 137 |
+
05/12/2026 12:02:32 - INFO - root - epochs: 3/5, steps: 4700/9590, func: 0.043737, pos: 0.032744, neg: 0.007529, 48%: 6h 35m 47s
|
| 138 |
+
05/12/2026 12:06:32 - INFO - root - epochs: 3/5, steps: 4750/9590, func: 0.04433, pos: 0.018101, neg: 0.00986, 49%: 6h 31m 42s
|
| 139 |
+
05/12/2026 12:10:37 - INFO - root - epochs: 3/5, steps: 4800/9590, func: 0.044883, pos: 0.031611, neg: 0.007338, 50%: 6h 27m 42s
|
| 140 |
+
05/12/2026 12:14:40 - INFO - root - epochs: 3/5, steps: 4850/9590, func: 0.043752, pos: 0.036319, neg: 0.010377, 50%: 6h 23m 39s
|
| 141 |
+
05/12/2026 12:18:40 - INFO - root - epochs: 3/5, steps: 4900/9590, func: 0.043916, pos: 0.026421, neg: 0.008696, 51%: 6h 19m 34s
|
| 142 |
+
05/12/2026 12:22:44 - INFO - root - epochs: 3/5, steps: 4950/9590, func: 0.043717, pos: 0.036711, neg: 0.014496, 51%: 6h 15m 32s
|
| 143 |
+
05/12/2026 12:26:45 - INFO - root - epochs: 3/5, steps: 5000/9590, func: 0.044367, pos: 0.041459, neg: 0.005121, 52%: 6h 11m 27s
|
| 144 |
+
05/12/2026 12:30:49 - INFO - root - epochs: 3/5, steps: 5050/9590, func: 0.044552, pos: 0.030175, neg: 0.00822, 52%: 6h 7m 26s
|
| 145 |
+
05/12/2026 12:34:59 - INFO - root - epochs: 3/5, steps: 5100/9590, func: 0.043727, pos: 0.031545, neg: 0.017947, 53%: 6h 3m 29s
|
| 146 |
+
05/12/2026 12:39:08 - INFO - root - epochs: 3/5, steps: 5150/9590, func: 0.044665, pos: 0.020976, neg: 0.013421, 53%: 5h 59m 33s
|
| 147 |
+
05/12/2026 12:43:13 - INFO - root - epochs: 3/5, steps: 5200/9590, func: 0.04315, pos: 0.035845, neg: 0.009733, 54%: 5h 55m 30s
|
| 148 |
+
05/12/2026 12:47:15 - INFO - root - epochs: 3/5, steps: 5250/9590, func: 0.04466, pos: 0.037906, neg: 0.009849, 54%: 5h 51m 27s
|
| 149 |
+
05/12/2026 12:51:14 - INFO - root - epochs: 3/5, steps: 5300/9590, func: 0.044784, pos: 0.034, neg: 0.015596, 55%: 5h 47m 21s
|
| 150 |
+
05/12/2026 12:55:14 - INFO - root - epochs: 3/5, steps: 5350/9590, func: 0.043455, pos: 0.028191, neg: 0.010957, 55%: 5h 43m 16s
|
| 151 |
+
05/12/2026 12:59:16 - INFO - root - epochs: 3/5, steps: 5400/9590, func: 0.042788, pos: 0.030653, neg: 0.011688, 56%: 5h 39m 12s
|
| 152 |
+
05/12/2026 13:03:23 - INFO - root - epochs: 3/5, steps: 5450/9590, func: 0.044157, pos: 0.025905, neg: 0.009461, 56%: 5h 35m 13s
|
| 153 |
+
05/12/2026 13:07:32 - INFO - root - epochs: 3/5, steps: 5500/9590, func: 0.045079, pos: 0.027724, neg: 0.009764, 57%: 5h 31m 15s
|
| 154 |
+
05/12/2026 13:11:38 - INFO - root - epochs: 3/5, steps: 5550/9590, func: 0.043324, pos: 0.032279, neg: 0.014533, 57%: 5h 27m 13s
|
| 155 |
+
05/12/2026 13:15:42 - INFO - root - epochs: 3/5, steps: 5600/9590, func: 0.043653, pos: 0.035368, neg: 0.005618, 58%: 5h 23m 11s
|
| 156 |
+
05/12/2026 13:19:45 - INFO - root - epochs: 3/5, steps: 5650/9590, func: 0.043204, pos: 0.024611, neg: 0.017919, 58%: 5h 19m 9s
|
| 157 |
+
05/12/2026 13:23:46 - INFO - root - epochs: 3/5, steps: 5700/9590, func: 0.04316, pos: 0.018795, neg: 0.010624, 59%: 5h 15m 4s
|
| 158 |
+
05/12/2026 13:27:48 - INFO - root - epochs: 3/5, steps: 5750/9590, func: 0.044979, pos: 0.025781, neg: 0.020653, 59%: 5h 11m 1s
|
| 159 |
+
05/12/2026 13:31:50 - INFO - root - epochs: 4/5, steps: 5800/9590, func: 0.037187, pos: 0.028788, neg: 0.006066, 60%: 5h 6m 58s
|
| 160 |
+
05/12/2026 13:35:52 - INFO - root - epochs: 4/5, steps: 5850/9590, func: 0.037183, pos: 0.01936, neg: 0.004128, 60%: 5h 2m 53s
|
| 161 |
+
05/12/2026 13:39:56 - INFO - root - epochs: 4/5, steps: 5900/9590, func: 0.036845, pos: 0.010634, neg: 0.006401, 61%: 4h 58m 51s
|
| 162 |
+
05/12/2026 13:43:55 - INFO - root - epochs: 4/5, steps: 5950/9590, func: 0.035898, pos: 0.014306, neg: 0.001101, 62%: 4h 54m 46s
|
| 163 |
+
05/12/2026 13:48:01 - INFO - root - epochs: 4/5, steps: 6000/9590, func: 0.036924, pos: 0.021029, neg: 0.003658, 62%: 4h 50m 45s
|
| 164 |
+
05/12/2026 13:52:07 - INFO - root - epochs: 4/5, steps: 6050/9590, func: 0.037266, pos: 0.026165, neg: 0.007137, 63%: 4h 46m 44s
|
| 165 |
+
05/12/2026 13:56:05 - INFO - root - epochs: 4/5, steps: 6100/9590, func: 0.03719, pos: 0.013228, neg: 0.004765, 63%: 4h 42m 38s
|
| 166 |
+
05/12/2026 14:00:08 - INFO - root - epochs: 4/5, steps: 6150/9590, func: 0.037471, pos: 0.011627, neg: 0.003193, 64%: 4h 38m 36s
|
| 167 |
+
05/12/2026 14:04:14 - INFO - root - epochs: 4/5, steps: 6200/9590, func: 0.037742, pos: 0.008895, neg: 0.007303, 64%: 4h 34m 34s
|
| 168 |
+
05/12/2026 14:08:17 - INFO - root - epochs: 4/5, steps: 6250/9590, func: 0.037754, pos: 0.011739, neg: 0.004153, 65%: 4h 30m 32s
|
| 169 |
+
05/12/2026 14:12:13 - INFO - root - epochs: 4/5, steps: 6300/9590, func: 0.036989, pos: 0.01173, neg: 0.008266, 65%: 4h 26m 25s
|
| 170 |
+
05/12/2026 14:16:20 - INFO - root - epochs: 4/5, steps: 6350/9590, func: 0.037897, pos: 0.01403, neg: 0.005232, 66%: 4h 22m 24s
|
| 171 |
+
05/12/2026 14:20:26 - INFO - root - epochs: 4/5, steps: 6400/9590, func: 0.037609, pos: 0.016425, neg: 0.002059, 66%: 4h 18m 23s
|
| 172 |
+
05/12/2026 14:24:32 - INFO - root - epochs: 4/5, steps: 6450/9590, func: 0.038468, pos: 0.019632, neg: 0.013029, 67%: 4h 14m 21s
|
| 173 |
+
05/12/2026 14:28:32 - INFO - root - epochs: 4/5, steps: 6500/9590, func: 0.038036, pos: 0.011861, neg: 0.003423, 67%: 4h 10m 17s
|
| 174 |
+
05/12/2026 14:32:33 - INFO - root - epochs: 4/5, steps: 6550/9590, func: 0.037247, pos: 0.027833, neg: 0.005179, 68%: 4h 6m 13s
|
| 175 |
+
05/12/2026 14:36:40 - INFO - root - epochs: 4/5, steps: 6600/9590, func: 0.037224, pos: 0.015212, neg: 0.006781, 68%: 4h 2m 12s
|
| 176 |
+
05/12/2026 14:40:41 - INFO - root - epochs: 4/5, steps: 6650/9590, func: 0.038324, pos: 0.015693, neg: 0.005974, 69%: 3h 58m 8s
|
| 177 |
+
05/12/2026 14:44:45 - INFO - root - epochs: 4/5, steps: 6700/9590, func: 0.039012, pos: 0.014201, neg: 0.00685, 69%: 3h 54m 6s
|
| 178 |
+
05/12/2026 14:48:52 - INFO - root - epochs: 4/5, steps: 6750/9590, func: 0.037505, pos: 0.010282, neg: 0.002384, 70%: 3h 50m 5s
|
| 179 |
+
05/12/2026 14:52:58 - INFO - root - epochs: 4/5, steps: 6800/9590, func: 0.037557, pos: 0.007854, neg: 0.003328, 70%: 3h 46m 3s
|
| 180 |
+
05/12/2026 14:56:55 - INFO - root - epochs: 4/5, steps: 6850/9590, func: 0.037521, pos: 0.012866, neg: 0.003266, 71%: 3h 41m 58s
|
| 181 |
+
05/12/2026 15:00:56 - INFO - root - epochs: 4/5, steps: 6900/9590, func: 0.037562, pos: 0.004959, neg: 0.003749, 71%: 3h 37m 54s
|
| 182 |
+
05/12/2026 15:04:57 - INFO - root - epochs: 4/5, steps: 6950/9590, func: 0.038498, pos: 0.011588, neg: 0.004565, 72%: 3h 33m 50s
|
| 183 |
+
05/12/2026 15:09:04 - INFO - root - epochs: 4/5, steps: 7000/9590, func: 0.037594, pos: 0.016966, neg: 0.006965, 72%: 3h 29m 49s
|
| 184 |
+
05/12/2026 15:13:12 - INFO - root - epochs: 4/5, steps: 7050/9590, func: 0.039776, pos: 0.013159, neg: 0.006337, 73%: 3h 25m 48s
|
| 185 |
+
05/12/2026 15:17:14 - INFO - root - epochs: 4/5, steps: 7100/9590, func: 0.037827, pos: 0.011383, neg: 0.004707, 74%: 3h 21m 44s
|
| 186 |
+
05/12/2026 15:21:16 - INFO - root - epochs: 4/5, steps: 7150/9590, func: 0.038144, pos: 0.024477, neg: 0.008781, 74%: 3h 17m 41s
|
| 187 |
+
05/12/2026 15:25:23 - INFO - root - epochs: 4/5, steps: 7200/9590, func: 0.039326, pos: 0.026087, neg: 0.008004, 75%: 3h 13m 39s
|
| 188 |
+
05/12/2026 15:29:22 - INFO - root - epochs: 4/5, steps: 7250/9590, func: 0.038138, pos: 0.012713, neg: 0.008708, 75%: 3h 9m 35s
|
| 189 |
+
05/12/2026 15:33:29 - INFO - root - epochs: 4/5, steps: 7300/9590, func: 0.037897, pos: 0.011754, neg: 0.003533, 76%: 3h 5m 34s
|
| 190 |
+
05/12/2026 15:37:28 - INFO - root - epochs: 4/5, steps: 7350/9590, func: 0.037133, pos: 0.011942, neg: 0.01366, 76%: 3h 1m 29s
|
| 191 |
+
05/12/2026 15:41:27 - INFO - root - epochs: 4/5, steps: 7400/9590, func: 0.03792, pos: 0.004823, neg: 0.002933, 77%: 2h 57m 25s
|
| 192 |
+
05/12/2026 15:45:36 - INFO - root - epochs: 4/5, steps: 7450/9590, func: 0.039068, pos: 0.013343, neg: 0.002967, 77%: 2h 53m 24s
|
| 193 |
+
05/12/2026 15:49:33 - INFO - root - epochs: 4/5, steps: 7500/9590, func: 0.03732, pos: 0.014963, neg: 0.00197, 78%: 2h 49m 19s
|
| 194 |
+
05/12/2026 15:53:39 - INFO - root - epochs: 4/5, steps: 7550/9590, func: 0.038959, pos: 0.005567, neg: 0.008477, 78%: 2h 45m 17s
|
| 195 |
+
05/12/2026 15:57:39 - INFO - root - epochs: 4/5, steps: 7600/9590, func: 0.036912, pos: 0.013687, neg: 0.004179, 79%: 2h 41m 14s
|
| 196 |
+
05/12/2026 16:01:41 - INFO - root - epochs: 4/5, steps: 7650/9590, func: 0.039019, pos: 0.014941, neg: 0.004959, 79%: 2h 37m 10s
|
| 197 |
+
05/12/2026 16:05:49 - INFO - root - epochs: 5/5, steps: 7700/9590, func: 0.033764, pos: 0.005057, neg: 0.003659, 80%: 2h 33m 9s
|
| 198 |
+
05/12/2026 16:09:54 - INFO - root - epochs: 5/5, steps: 7750/9590, func: 0.029872, pos: 0.003296, neg: 0.00179, 80%: 2h 29m 6s
|
| 199 |
+
05/12/2026 16:13:57 - INFO - root - epochs: 5/5, steps: 7800/9590, func: 0.030226, pos: 0.004068, neg: 0.000308, 81%: 2h 25m 3s
|
| 200 |
+
05/12/2026 16:17:56 - INFO - root - epochs: 5/5, steps: 7850/9590, func: 0.028933, pos: 0.014822, neg: 0.004078, 81%: 2h 20m 59s
|
| 201 |
+
05/12/2026 16:21:59 - INFO - root - epochs: 5/5, steps: 7900/9590, func: 0.030283, pos: 0.003466, neg: 0.003448, 82%: 2h 16m 56s
|
| 202 |
+
05/12/2026 16:26:04 - INFO - root - epochs: 5/5, steps: 7950/9590, func: 0.029845, pos: 0.007248, neg: 0.001106, 82%: 2h 12m 54s
|
| 203 |
+
05/12/2026 16:30:08 - INFO - root - epochs: 5/5, steps: 8000/9590, func: 0.031504, pos: 0.005889, neg: 0.001085, 83%: 2h 8m 51s
|
| 204 |
+
05/12/2026 16:34:14 - INFO - root - epochs: 5/5, steps: 8050/9590, func: 0.031164, pos: 0.003975, neg: 0.00115, 83%: 2h 4m 49s
|
| 205 |
+
05/12/2026 16:38:12 - INFO - root - epochs: 5/5, steps: 8100/9590, func: 0.030783, pos: 0.003097, neg: 0.001654, 84%: 2h 0m 45s
|
| 206 |
+
05/12/2026 16:42:11 - INFO - root - epochs: 5/5, steps: 8150/9590, func: 0.030095, pos: 0.003739, neg: 0.001491, 84%: 1h 56m 41s
|
| 207 |
+
05/12/2026 16:46:10 - INFO - root - epochs: 5/5, steps: 8200/9590, func: 0.029816, pos: 0.007938, neg: 0.000674, 85%: 1h 52m 37s
|
| 208 |
+
05/12/2026 16:50:14 - INFO - root - epochs: 5/5, steps: 8250/9590, func: 0.031022, pos: 0.002691, neg: 0.002459, 86%: 1h 48m 35s
|
| 209 |
+
05/12/2026 16:54:18 - INFO - root - epochs: 5/5, steps: 8300/9590, func: 0.030573, pos: 0.006589, neg: 0.00105, 86%: 1h 44m 32s
|
| 210 |
+
05/12/2026 16:58:19 - INFO - root - epochs: 5/5, steps: 8350/9590, func: 0.030611, pos: 0.00893, neg: 0.001367, 87%: 1h 40m 29s
|
| 211 |
+
05/12/2026 17:02:25 - INFO - root - epochs: 5/5, steps: 8400/9590, func: 0.03111, pos: 0.006094, neg: 0.003428, 87%: 1h 36m 26s
|
| 212 |
+
05/12/2026 17:06:28 - INFO - root - epochs: 5/5, steps: 8450/9590, func: 0.031174, pos: 0.006896, neg: 0.001139, 88%: 1h 32m 23s
|
| 213 |
+
05/12/2026 17:10:33 - INFO - root - epochs: 5/5, steps: 8500/9590, func: 0.03143, pos: 0.010165, neg: 0.001063, 88%: 1h 28m 21s
|
| 214 |
+
05/12/2026 17:14:33 - INFO - root - epochs: 5/5, steps: 8550/9590, func: 0.02977, pos: 0.001661, neg: 0.000399, 89%: 1h 24m 17s
|
| 215 |
+
05/12/2026 17:18:38 - INFO - root - epochs: 5/5, steps: 8600/9590, func: 0.030976, pos: 0.002311, neg: 0.001703, 89%: 1h 20m 14s
|
| 216 |
+
05/12/2026 17:22:42 - INFO - root - epochs: 5/5, steps: 8650/9590, func: 0.029891, pos: 0.003819, neg: 0.002301, 90%: 1h 16m 12s
|
| 217 |
+
05/12/2026 17:26:45 - INFO - root - epochs: 5/5, steps: 8700/9590, func: 0.030551, pos: 0.001962, neg: 0.000672, 90%: 1h 12m 9s
|
| 218 |
+
05/12/2026 17:30:50 - INFO - root - epochs: 5/5, steps: 8750/9590, func: 0.03133, pos: 0.004639, neg: 0.015719, 91%: 1h 8m 6s
|
| 219 |
+
05/12/2026 17:34:52 - INFO - root - epochs: 5/5, steps: 8800/9590, func: 0.030877, pos: 0.003829, neg: 0.002129, 91%: 1h 4m 3s
|
| 220 |
+
05/12/2026 17:38:52 - INFO - root - epochs: 5/5, steps: 8850/9590, func: 0.029963, pos: 0.002877, neg: 0.000992, 92%: 1h 0m 0s
|
| 221 |
+
05/12/2026 17:42:59 - INFO - root - epochs: 5/5, steps: 8900/9590, func: 0.030816, pos: 0.003824, neg: 0.000756, 92%: 0h 55m 57s
|
| 222 |
+
05/12/2026 17:47:04 - INFO - root - epochs: 5/5, steps: 8950/9590, func: 0.031247, pos: 0.001944, neg: 0.005025, 93%: 0h 51m 54s
|
| 223 |
+
05/12/2026 17:51:06 - INFO - root - epochs: 5/5, steps: 9000/9590, func: 0.030236, pos: 0.004118, neg: 0.00032, 93%: 0h 47m 51s
|
| 224 |
+
05/12/2026 17:55:06 - INFO - root - epochs: 5/5, steps: 9050/9590, func: 0.030045, pos: 0.003925, neg: 0.006625, 94%: 0h 43m 48s
|
| 225 |
+
05/12/2026 17:59:11 - INFO - root - epochs: 5/5, steps: 9100/9590, func: 0.030309, pos: 0.007086, neg: 0.001351, 94%: 0h 39m 45s
|
| 226 |
+
05/12/2026 18:03:12 - INFO - root - epochs: 5/5, steps: 9150/9590, func: 0.030451, pos: 0.001467, neg: 0.004338, 95%: 0h 35m 42s
|
| 227 |
+
05/12/2026 18:07:17 - INFO - root - epochs: 5/5, steps: 9200/9590, func: 0.030429, pos: 0.007513, neg: 0.003631, 95%: 0h 31m 39s
|
| 228 |
+
05/12/2026 18:11:21 - INFO - root - epochs: 5/5, steps: 9250/9590, func: 0.030666, pos: 0.003335, neg: 0.003133, 96%: 0h 27m 37s
|
| 229 |
+
05/12/2026 18:15:25 - INFO - root - epochs: 5/5, steps: 9300/9590, func: 0.030095, pos: 0.006037, neg: 0.001205, 96%: 0h 23m 34s
|
| 230 |
+
05/12/2026 18:19:29 - INFO - root - epochs: 5/5, steps: 9350/9590, func: 0.030603, pos: 0.005101, neg: 0.001488, 97%: 0h 19m 31s
|
| 231 |
+
05/12/2026 18:23:32 - INFO - root - epochs: 5/5, steps: 9400/9590, func: 0.031185, pos: 0.00453, neg: 0.000905, 98%: 0h 15m 28s
|
| 232 |
+
05/12/2026 18:27:35 - INFO - root - epochs: 5/5, steps: 9450/9590, func: 0.031027, pos: 0.005386, neg: 0.00126, 98%: 0h 11m 25s
|
| 233 |
+
05/12/2026 18:31:37 - INFO - root - epochs: 5/5, steps: 9500/9590, func: 0.030854, pos: 0.007732, neg: 0.00126, 99%: 0h 7m 22s
|
| 234 |
+
05/12/2026 18:35:37 - INFO - root - epochs: 5/5, steps: 9550/9590, func: 0.031091, pos: 0.004284, neg: 0.000373, 99%: 0h 3m 19s
|
| 235 |
+
05/12/2026 18:47:11 - INFO - root - final eval loss: func: 0.051988, pos: 0.0564, neg: 0.048031
|
| 236 |
+
05/12/2026 18:47:11 - INFO - root - Saving model checkpoint to ../trained/codellama-7b-lora-safecoder/checkpoint-last
|
deepseek-coder-1.3b-lora-safecoder/checkpoint-last/README.md
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: deepseek-ai/deepseek-coder-1.3b-base
|
| 3 |
+
library_name: peft
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Model Card for Model ID
|
| 7 |
+
|
| 8 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
## Model Details
|
| 13 |
+
|
| 14 |
+
### Model Description
|
| 15 |
+
|
| 16 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
- **Developed by:** [More Information Needed]
|
| 21 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 22 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 23 |
+
- **Model type:** [More Information Needed]
|
| 24 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 25 |
+
- **License:** [More Information Needed]
|
| 26 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 27 |
+
|
| 28 |
+
### Model Sources [optional]
|
| 29 |
+
|
| 30 |
+
<!-- Provide the basic links for the model. -->
|
| 31 |
+
|
| 32 |
+
- **Repository:** [More Information Needed]
|
| 33 |
+
- **Paper [optional]:** [More Information Needed]
|
| 34 |
+
- **Demo [optional]:** [More Information Needed]
|
| 35 |
+
|
| 36 |
+
## Uses
|
| 37 |
+
|
| 38 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 39 |
+
|
| 40 |
+
### Direct Use
|
| 41 |
+
|
| 42 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 43 |
+
|
| 44 |
+
[More Information Needed]
|
| 45 |
+
|
| 46 |
+
### Downstream Use [optional]
|
| 47 |
+
|
| 48 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 49 |
+
|
| 50 |
+
[More Information Needed]
|
| 51 |
+
|
| 52 |
+
### Out-of-Scope Use
|
| 53 |
+
|
| 54 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 55 |
+
|
| 56 |
+
[More Information Needed]
|
| 57 |
+
|
| 58 |
+
## Bias, Risks, and Limitations
|
| 59 |
+
|
| 60 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 61 |
+
|
| 62 |
+
[More Information Needed]
|
| 63 |
+
|
| 64 |
+
### Recommendations
|
| 65 |
+
|
| 66 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 67 |
+
|
| 68 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 69 |
+
|
| 70 |
+
## How to Get Started with the Model
|
| 71 |
+
|
| 72 |
+
Use the code below to get started with the model.
|
| 73 |
+
|
| 74 |
+
[More Information Needed]
|
| 75 |
+
|
| 76 |
+
## Training Details
|
| 77 |
+
|
| 78 |
+
### Training Data
|
| 79 |
+
|
| 80 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 81 |
+
|
| 82 |
+
[More Information Needed]
|
| 83 |
+
|
| 84 |
+
### Training Procedure
|
| 85 |
+
|
| 86 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 87 |
+
|
| 88 |
+
#### Preprocessing [optional]
|
| 89 |
+
|
| 90 |
+
[More Information Needed]
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
#### Training Hyperparameters
|
| 94 |
+
|
| 95 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 96 |
+
|
| 97 |
+
#### Speeds, Sizes, Times [optional]
|
| 98 |
+
|
| 99 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 100 |
+
|
| 101 |
+
[More Information Needed]
|
| 102 |
+
|
| 103 |
+
## Evaluation
|
| 104 |
+
|
| 105 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 106 |
+
|
| 107 |
+
### Testing Data, Factors & Metrics
|
| 108 |
+
|
| 109 |
+
#### Testing Data
|
| 110 |
+
|
| 111 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 112 |
+
|
| 113 |
+
[More Information Needed]
|
| 114 |
+
|
| 115 |
+
#### Factors
|
| 116 |
+
|
| 117 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 118 |
+
|
| 119 |
+
[More Information Needed]
|
| 120 |
+
|
| 121 |
+
#### Metrics
|
| 122 |
+
|
| 123 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 124 |
+
|
| 125 |
+
[More Information Needed]
|
| 126 |
+
|
| 127 |
+
### Results
|
| 128 |
+
|
| 129 |
+
[More Information Needed]
|
| 130 |
+
|
| 131 |
+
#### Summary
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
## Model Examination [optional]
|
| 136 |
+
|
| 137 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 138 |
+
|
| 139 |
+
[More Information Needed]
|
| 140 |
+
|
| 141 |
+
## Environmental Impact
|
| 142 |
+
|
| 143 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 144 |
+
|
| 145 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 146 |
+
|
| 147 |
+
- **Hardware Type:** [More Information Needed]
|
| 148 |
+
- **Hours used:** [More Information Needed]
|
| 149 |
+
- **Cloud Provider:** [More Information Needed]
|
| 150 |
+
- **Compute Region:** [More Information Needed]
|
| 151 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 152 |
+
|
| 153 |
+
## Technical Specifications [optional]
|
| 154 |
+
|
| 155 |
+
### Model Architecture and Objective
|
| 156 |
+
|
| 157 |
+
[More Information Needed]
|
| 158 |
+
|
| 159 |
+
### Compute Infrastructure
|
| 160 |
+
|
| 161 |
+
[More Information Needed]
|
| 162 |
+
|
| 163 |
+
#### Hardware
|
| 164 |
+
|
| 165 |
+
[More Information Needed]
|
| 166 |
+
|
| 167 |
+
#### Software
|
| 168 |
+
|
| 169 |
+
[More Information Needed]
|
| 170 |
+
|
| 171 |
+
## Citation [optional]
|
| 172 |
+
|
| 173 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 174 |
+
|
| 175 |
+
**BibTeX:**
|
| 176 |
+
|
| 177 |
+
[More Information Needed]
|
| 178 |
+
|
| 179 |
+
**APA:**
|
| 180 |
+
|
| 181 |
+
[More Information Needed]
|
| 182 |
+
|
| 183 |
+
## Glossary [optional]
|
| 184 |
+
|
| 185 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 186 |
+
|
| 187 |
+
[More Information Needed]
|
| 188 |
+
|
| 189 |
+
## More Information [optional]
|
| 190 |
+
|
| 191 |
+
[More Information Needed]
|
| 192 |
+
|
| 193 |
+
## Model Card Authors [optional]
|
| 194 |
+
|
| 195 |
+
[More Information Needed]
|
| 196 |
+
|
| 197 |
+
## Model Card Contact
|
| 198 |
+
|
| 199 |
+
[More Information Needed]
|
| 200 |
+
### Framework versions
|
| 201 |
+
|
| 202 |
+
- PEFT 0.8.2
|
deepseek-coder-1.3b-lora-safecoder/checkpoint-last/adapter_config.json
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alpha_pattern": {},
|
| 3 |
+
"auto_mapping": null,
|
| 4 |
+
"base_model_name_or_path": "deepseek-ai/deepseek-coder-1.3b-base",
|
| 5 |
+
"bias": "none",
|
| 6 |
+
"fan_in_fan_out": false,
|
| 7 |
+
"inference_mode": true,
|
| 8 |
+
"init_lora_weights": true,
|
| 9 |
+
"layers_pattern": null,
|
| 10 |
+
"layers_to_transform": null,
|
| 11 |
+
"loftq_config": {},
|
| 12 |
+
"lora_alpha": 32,
|
| 13 |
+
"lora_dropout": 0.1,
|
| 14 |
+
"megatron_config": null,
|
| 15 |
+
"megatron_core": "megatron.core",
|
| 16 |
+
"modules_to_save": null,
|
| 17 |
+
"peft_type": "LORA",
|
| 18 |
+
"r": 16,
|
| 19 |
+
"rank_pattern": {},
|
| 20 |
+
"revision": null,
|
| 21 |
+
"target_modules": [
|
| 22 |
+
"k_proj",
|
| 23 |
+
"down_proj",
|
| 24 |
+
"o_proj",
|
| 25 |
+
"q_proj",
|
| 26 |
+
"lm_head",
|
| 27 |
+
"v_proj",
|
| 28 |
+
"gate_proj",
|
| 29 |
+
"up_proj"
|
| 30 |
+
],
|
| 31 |
+
"task_type": "CAUSAL_LM",
|
| 32 |
+
"use_rslora": false
|
| 33 |
+
}
|
deepseek-coder-1.3b-lora-safecoder/checkpoint-last/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e345cd52ab595ae60fbe3ee0513ac80887823c502525b8d0d66876a928d52c5a
|
| 3 |
+
size 324515888
|
deepseek-coder-1.3b-lora-safecoder/checkpoint-last/special_tokens_map.json
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token": {
|
| 3 |
+
"content": "<｜begin▁of▁sentence｜>",
|
| 4 |
+
"lstrip": false,
|
| 5 |
+
"normalized": true,
|
| 6 |
+
"rstrip": false,
|
| 7 |
+
"single_word": false
|
| 8 |
+
},
|
| 9 |
+
"eos_token": {
|
| 10 |
+
"content": "<｜end▁of▁sentence｜>",
|
| 11 |
+
"lstrip": false,
|
| 12 |
+
"normalized": true,
|
| 13 |
+
"rstrip": false,
|
| 14 |
+
"single_word": false
|
| 15 |
+
},
|
| 16 |
+
"pad_token": {
|
| 17 |
+
"content": "<｜end▁of▁sentence｜>",
|
| 18 |
+
"lstrip": false,
|
| 19 |
+
"normalized": true,
|
| 20 |
+
"rstrip": false,
|
| 21 |
+
"single_word": false
|
| 22 |
+
}
|
| 23 |
+
}
|
deepseek-coder-1.3b-lora-safecoder/checkpoint-last/tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
deepseek-coder-1.3b-lora-safecoder/checkpoint-last/tokenizer_config.json
ADDED
|
@@ -0,0 +1,194 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_bos_token": true,
|
| 3 |
+
"add_eos_token": false,
|
| 4 |
+
"add_prefix_space": null,
|
| 5 |
+
"added_tokens_decoder": {
|
| 6 |
+
"32000": {
|
| 7 |
+
"content": "õ",
|
| 8 |
+
"lstrip": false,
|
| 9 |
+
"normalized": true,
|
| 10 |
+
"rstrip": false,
|
| 11 |
+
"single_word": false,
|
| 12 |
+
"special": false
|
| 13 |
+
},
|
| 14 |
+
"32001": {
|
| 15 |
+
"content": "÷",
|
| 16 |
+
"lstrip": false,
|
| 17 |
+
"normalized": true,
|
| 18 |
+
"rstrip": false,
|
| 19 |
+
"single_word": false,
|
| 20 |
+
"special": false
|
| 21 |
+
},
|
| 22 |
+
"32002": {
|
| 23 |
+
"content": "Á",
|
| 24 |
+
"lstrip": false,
|
| 25 |
+
"normalized": true,
|
| 26 |
+
"rstrip": false,
|
| 27 |
+
"single_word": false,
|
| 28 |
+
"special": false
|
| 29 |
+
},
|
| 30 |
+
"32003": {
|
| 31 |
+
"content": "ý",
|
| 32 |
+
"lstrip": false,
|
| 33 |
+
"normalized": true,
|
| 34 |
+
"rstrip": false,
|
| 35 |
+
"single_word": false,
|
| 36 |
+
"special": false
|
| 37 |
+
},
|
| 38 |
+
"32004": {
|
| 39 |
+
"content": "À",
|
| 40 |
+
"lstrip": false,
|
| 41 |
+
"normalized": true,
|
| 42 |
+
"rstrip": false,
|
| 43 |
+
"single_word": false,
|
| 44 |
+
"special": false
|
| 45 |
+
},
|
| 46 |
+
"32005": {
|
| 47 |
+
"content": "ÿ",
|
| 48 |
+
"lstrip": false,
|
| 49 |
+
"normalized": true,
|
| 50 |
+
"rstrip": false,
|
| 51 |
+
"single_word": false,
|
| 52 |
+
"special": false
|
| 53 |
+
},
|
| 54 |
+
"32006": {
|
| 55 |
+
"content": "ø",
|
| 56 |
+
"lstrip": false,
|
| 57 |
+
"normalized": true,
|
| 58 |
+
"rstrip": false,
|
| 59 |
+
"single_word": false,
|
| 60 |
+
"special": false
|
| 61 |
+
},
|
| 62 |
+
"32007": {
|
| 63 |
+
"content": "ú",
|
| 64 |
+
"lstrip": false,
|
| 65 |
+
"normalized": true,
|
| 66 |
+
"rstrip": false,
|
| 67 |
+
"single_word": false,
|
| 68 |
+
"special": false
|
| 69 |
+
},
|
| 70 |
+
"32008": {
|
| 71 |
+
"content": "þ",
|
| 72 |
+
"lstrip": false,
|
| 73 |
+
"normalized": true,
|
| 74 |
+
"rstrip": false,
|
| 75 |
+
"single_word": false,
|
| 76 |
+
"special": false
|
| 77 |
+
},
|
| 78 |
+
"32009": {
|
| 79 |
+
"content": "ü",
|
| 80 |
+
"lstrip": false,
|
| 81 |
+
"normalized": true,
|
| 82 |
+
"rstrip": false,
|
| 83 |
+
"single_word": false,
|
| 84 |
+
"special": false
|
| 85 |
+
},
|
| 86 |
+
"32010": {
|
| 87 |
+
"content": "ù",
|
| 88 |
+
"lstrip": false,
|
| 89 |
+
"normalized": true,
|
| 90 |
+
"rstrip": false,
|
| 91 |
+
"single_word": false,
|
| 92 |
+
"special": false
|
| 93 |
+
},
|
| 94 |
+
"32011": {
|
| 95 |
+
"content": "ö",
|
| 96 |
+
"lstrip": false,
|
| 97 |
+
"normalized": true,
|
| 98 |
+
"rstrip": false,
|
| 99 |
+
"single_word": false,
|
| 100 |
+
"special": false
|
| 101 |
+
},
|
| 102 |
+
"32012": {
|
| 103 |
+
"content": "û",
|
| 104 |
+
"lstrip": false,
|
| 105 |
+
"normalized": true,
|
| 106 |
+
"rstrip": false,
|
| 107 |
+
"single_word": false,
|
| 108 |
+
"special": false
|
| 109 |
+
},
|
| 110 |
+
"32013": {
|
| 111 |
+
"content": "<｜begin▁of▁sentence｜>",
|
| 112 |
+
"lstrip": false,
|
| 113 |
+
"normalized": true,
|
| 114 |
+
"rstrip": false,
|
| 115 |
+
"single_word": false,
|
| 116 |
+
"special": true
|
| 117 |
+
},
|
| 118 |
+
"32014": {
|
| 119 |
+
"content": "<｜end▁of▁sentence｜>",
|
| 120 |
+
"lstrip": false,
|
| 121 |
+
"normalized": true,
|
| 122 |
+
"rstrip": false,
|
| 123 |
+
"single_word": false,
|
| 124 |
+
"special": true
|
| 125 |
+
},
|
| 126 |
+
"32015": {
|
| 127 |
+
"content": "<｜fim▁hole｜>",
|
| 128 |
+
"lstrip": false,
|
| 129 |
+
"normalized": true,
|
| 130 |
+
"rstrip": false,
|
| 131 |
+
"single_word": false,
|
| 132 |
+
"special": false
|
| 133 |
+
},
|
| 134 |
+
"32016": {
|
| 135 |
+
"content": "<｜fim▁begin｜>",
|
| 136 |
+
"lstrip": false,
|
| 137 |
+
"normalized": true,
|
| 138 |
+
"rstrip": false,
|
| 139 |
+
"single_word": false,
|
| 140 |
+
"special": false
|
| 141 |
+
},
|
| 142 |
+
"32017": {
|
| 143 |
+
"content": "<｜fim▁end｜>",
|
| 144 |
+
"lstrip": false,
|
| 145 |
+
"normalized": true,
|
| 146 |
+
"rstrip": false,
|
| 147 |
+
"single_word": false,
|
| 148 |
+
"special": false
|
| 149 |
+
},
|
| 150 |
+
"32018": {
|
| 151 |
+
"content": "<pad>",
|
| 152 |
+
"lstrip": false,
|
| 153 |
+
"normalized": true,
|
| 154 |
+
"rstrip": false,
|
| 155 |
+
"single_word": false,
|
| 156 |
+
"special": false
|
| 157 |
+
},
|
| 158 |
+
"32019": {
|
| 159 |
+
"content": "<|User|>",
|
| 160 |
+
"lstrip": false,
|
| 161 |
+
"normalized": true,
|
| 162 |
+
"rstrip": false,
|
| 163 |
+
"single_word": false,
|
| 164 |
+
"special": false
|
| 165 |
+
},
|
| 166 |
+
"32020": {
|
| 167 |
+
"content": "<|Assistant|>",
|
| 168 |
+
"lstrip": false,
|
| 169 |
+
"normalized": true,
|
| 170 |
+
"rstrip": false,
|
| 171 |
+
"single_word": false,
|
| 172 |
+
"special": false
|
| 173 |
+
},
|
| 174 |
+
"32021": {
|
| 175 |
+
"content": "<|EOT|>",
|
| 176 |
+
"lstrip": false,
|
| 177 |
+
"normalized": true,
|
| 178 |
+
"rstrip": false,
|
| 179 |
+
"single_word": false,
|
| 180 |
+
"special": false
|
| 181 |
+
}
|
| 182 |
+
},
|
| 183 |
+
"bos_token": "<｜begin▁of▁sentence｜>",
|
| 184 |
+
"clean_up_tokenization_spaces": false,
|
| 185 |
+
"eos_token": "<｜end▁of▁sentence｜>",
|
| 186 |
+
"extra_special_tokens": {},
|
| 187 |
+
"legacy": true,
|
| 188 |
+
"model_max_length": 16384,
|
| 189 |
+
"pad_token": "<｜end▁of▁sentence｜>",
|
| 190 |
+
"sp_model_kwargs": {},
|
| 191 |
+
"tokenizer_class": "LlamaTokenizerFast",
|
| 192 |
+
"unk_token": null,
|
| 193 |
+
"use_default_system_prompt": false
|
| 194 |
+
}
|
deepseek-coder-1.3b-lora-safecoder/train.log
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
05/12/2026 02:58:53 - INFO - accelerate.utils.modeling - We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can set `max_memory` in to a higher value to use more memory (at your own risk).
|
| 2 |
+
05/12/2026 02:59:58 - INFO - root - number of sec samples before upsampling: 1810
|
| 3 |
+
05/12/2026 02:59:58 - INFO - root - number of sec samples after upsampling: 2415
|
| 4 |
+
05/12/2026 03:00:06 - INFO - root - Training args Namespace(output_name='deepseek-coder-1.3b-lora-safecoder', datasets=['evol', 'sec-desc', 'sec-new-desc'], pretrain_name='deepseek-coder-1.3b', loss_weight=1.0, sven=False, num_train_epochs=2, learning_rate=2e-05, max_num_tokens=1024, batch_size=1, grad_acc_steps=16, weight_decay=0.01, adam_epsilon=1e-08, warmup_steps=0, max_grad_norm=1.0, dropout=0.1, kl_loss_weight=0, exclude_neg=False, no_weights=False, lora=True, r=16, lora_alpha=32, lora_dropout=0.1, sampling_size=20, sampling_method='minority', cwes=['all'], langs=['all'], logging_steps=50, save_epochs=10, seed=2, data_dir='../data_train_val', model_dir='../trained/', output_dir='../trained/deepseek-coder-1.3b-lora-safecoder', logger=<RootLogger root (INFO)>)
|
| 5 |
+
05/12/2026 03:00:06 - INFO - root - ***** Running training *****
|
| 6 |
+
05/12/2026 03:00:06 - INFO - root - Num samples = 30979
|
| 7 |
+
05/12/2026 03:00:06 - INFO - root - Num epoch = 2
|
| 8 |
+
05/12/2026 03:00:06 - INFO - root - Batch size= 1
|
| 9 |
+
05/12/2026 03:00:06 - INFO - root - Total batch size (w. accumulation) = 16
|
| 10 |
+
05/12/2026 03:00:06 - INFO - root - Gradient Accumulation steps = 16
|
| 11 |
+
05/12/2026 03:00:06 - INFO - root - Total optimization steps = 3872
|
| 12 |
+
05/12/2026 03:00:06 - INFO - root - Num val samples = 3371
|
| 13 |
+
05/12/2026 03:00:06 - INFO - root - Num parameters = 1361049952
|
| 14 |
+
05/12/2026 03:00:06 - INFO - root - Num trainable parameters = 15536480
|
| 15 |
+
05/12/2026 03:02:57 - INFO - root - epochs: 1/2, steps: 50/3872, func: 0.058498, pos: 0.063538, neg: 0.177169, 1%: 3h 38m 54s
|
| 16 |
+
05/12/2026 03:05:50 - INFO - root - epochs: 1/2, steps: 100/3872, func: 0.055839, pos: 0.074381, neg: 0.136812, 2%: 3h 36m 27s
|
| 17 |
+
05/12/2026 03:08:41 - INFO - root - epochs: 1/2, steps: 150/3872, func: 0.054501, pos: 0.079344, neg: 0.138904, 3%: 3h 32m 59s
|
| 18 |
+
05/12/2026 03:11:33 - INFO - root - epochs: 1/2, steps: 200/3872, func: 0.054391, pos: 0.069465, neg: 0.120665, 5%: 3h 30m 31s
|
| 19 |
+
05/12/2026 03:14:23 - INFO - root - epochs: 1/2, steps: 250/3872, func: 0.055671, pos: 0.080729, neg: 0.104012, 6%: 3h 26m 58s
|
| 20 |
+
05/12/2026 03:17:12 - INFO - root - epochs: 1/2, steps: 300/3872, func: 0.053413, pos: 0.065483, neg: 0.067479, 7%: 3h 23m 48s
|
| 21 |
+
05/12/2026 03:20:01 - INFO - root - epochs: 1/2, steps: 350/3872, func: 0.055215, pos: 0.075811, neg: 0.051169, 9%: 3h 20m 35s
|
| 22 |
+
05/12/2026 03:22:55 - INFO - root - epochs: 1/2, steps: 400/3872, func: 0.053353, pos: 0.079163, neg: 0.046558, 10%: 3h 18m 6s
|
| 23 |
+
05/12/2026 03:25:48 - INFO - root - epochs: 1/2, steps: 450/3872, func: 0.05248, pos: 0.080903, neg: 0.050838, 11%: 3h 15m 31s
|
| 24 |
+
05/12/2026 03:28:41 - INFO - root - epochs: 1/2, steps: 500/3872, func: 0.052438, pos: 0.07525, neg: 0.06079, 12%: 3h 12m 53s
|
| 25 |
+
05/12/2026 03:31:31 - INFO - root - epochs: 1/2, steps: 550/3872, func: 0.052956, pos: 0.064179, neg: 0.037968, 14%: 3h 9m 48s
|
| 26 |
+
05/12/2026 03:34:25 - INFO - root - epochs: 1/2, steps: 600/3872, func: 0.052458, pos: 0.085765, neg: 0.048997, 15%: 3h 7m 7s
|
| 27 |
+
05/12/2026 03:37:14 - INFO - root - epochs: 1/2, steps: 650/3872, func: 0.051482, pos: 0.065869, neg: 0.041883, 16%: 3h 4m 7s
|
| 28 |
+
05/12/2026 03:40:10 - INFO - root - epochs: 1/2, steps: 700/3872, func: 0.053914, pos: 0.080435, neg: 0.027877, 18%: 3h 1m 40s
|
| 29 |
+
05/12/2026 03:43:01 - INFO - root - epochs: 1/2, steps: 750/3872, func: 0.053491, pos: 0.072454, neg: 0.035127, 19%: 2h 58m 42s
|
| 30 |
+
05/12/2026 03:45:50 - INFO - root - epochs: 1/2, steps: 800/3872, func: 0.053087, pos: 0.069237, neg: 0.031153, 20%: 2h 55m 39s
|
| 31 |
+
05/12/2026 03:48:40 - INFO - root - epochs: 1/2, steps: 850/3872, func: 0.052351, pos: 0.0773, neg: 0.033514, 21%: 2h 52m 47s
|
| 32 |
+
05/12/2026 03:51:31 - INFO - root - epochs: 1/2, steps: 900/3872, func: 0.052265, pos: 0.089922, neg: 0.029248, 23%: 2h 49m 53s
|
| 33 |
+
05/12/2026 03:54:22 - INFO - root - epochs: 1/2, steps: 950/3872, func: 0.052332, pos: 0.066269, neg: 0.033777, 24%: 2h 46m 57s
|
| 34 |
+
05/12/2026 03:57:13 - INFO - root - epochs: 1/2, steps: 1000/3872, func: 0.052323, pos: 0.056476, neg: 0.039315, 25%: 2h 44m 6s
|
| 35 |
+
05/12/2026 04:00:04 - INFO - root - epochs: 1/2, steps: 1050/3872, func: 0.052345, pos: 0.079464, neg: 0.029279, 27%: 2h 41m 14s
|
| 36 |
+
05/12/2026 04:03:00 - INFO - root - epochs: 1/2, steps: 1100/3872, func: 0.0532, pos: 0.075645, neg: 0.028954, 28%: 2h 38m 34s
|
| 37 |
+
05/12/2026 04:05:52 - INFO - root - epochs: 1/2, steps: 1150/3872, func: 0.051499, pos: 0.053088, neg: 0.037814, 29%: 2h 35m 44s
|
| 38 |
+
05/12/2026 04:08:43 - INFO - root - epochs: 1/2, steps: 1200/3872, func: 0.051378, pos: 0.066166, neg: 0.03289, 30%: 2h 32m 52s
|
| 39 |
+
05/12/2026 04:11:36 - INFO - root - epochs: 1/2, steps: 1250/3872, func: 0.052493, pos: 0.075589, neg: 0.030921, 32%: 2h 30m 1s
|
| 40 |
+
05/12/2026 04:14:26 - INFO - root - epochs: 1/2, steps: 1300/3872, func: 0.051639, pos: 0.062583, neg: 0.037883, 33%: 2h 27m 8s
|
| 41 |
+
05/12/2026 04:17:13 - INFO - root - epochs: 1/2, steps: 1350/3872, func: 0.050644, pos: 0.065618, neg: 0.034679, 34%: 2h 24m 7s
|
| 42 |
+
05/12/2026 04:20:07 - INFO - root - epochs: 1/2, steps: 1400/3872, func: 0.052182, pos: 0.058463, neg: 0.035739, 36%: 2h 21m 21s
|
| 43 |
+
05/12/2026 04:22:59 - INFO - root - epochs: 1/2, steps: 1450/3872, func: 0.052085, pos: 0.059989, neg: 0.034663, 37%: 2h 18m 29s
|
| 44 |
+
05/12/2026 04:25:54 - INFO - root - epochs: 1/2, steps: 1500/3872, func: 0.052605, pos: 0.067227, neg: 0.030407, 38%: 2h 15m 44s
|
| 45 |
+
05/12/2026 04:28:46 - INFO - root - epochs: 1/2, steps: 1550/3872, func: 0.05215, pos: 0.053883, neg: 0.039369, 40%: 2h 12m 53s
|
| 46 |
+
05/12/2026 04:31:37 - INFO - root - epochs: 1/2, steps: 1600/3872, func: 0.053258, pos: 0.069293, neg: 0.033501, 41%: 2h 10m 1s
|
| 47 |
+
05/12/2026 04:34:29 - INFO - root - epochs: 1/2, steps: 1650/3872, func: 0.051802, pos: 0.098566, neg: 0.023335, 42%: 2h 7m 10s
|
| 48 |
+
05/12/2026 04:37:23 - INFO - root - epochs: 1/2, steps: 1700/3872, func: 0.053117, pos: 0.071336, neg: 0.027185, 43%: 2h 4m 21s
|
| 49 |
+
05/12/2026 04:40:13 - INFO - root - epochs: 1/2, steps: 1750/3872, func: 0.0523, pos: 0.059277, neg: 0.023508, 45%: 2h 1m 28s
|
| 50 |
+
05/12/2026 04:43:06 - INFO - root - epochs: 1/2, steps: 1800/3872, func: 0.051384, pos: 0.0708, neg: 0.025847, 46%: 1h 58m 37s
|
| 51 |
+
05/12/2026 04:45:56 - INFO - root - epochs: 1/2, steps: 1850/3872, func: 0.051056, pos: 0.058926, neg: 0.023186, 47%: 1h 55m 43s
|
| 52 |
+
05/12/2026 04:48:43 - INFO - root - epochs: 1/2, steps: 1900/3872, func: 0.051681, pos: 0.052175, neg: 0.026962, 49%: 1h 52m 47s
|
| 53 |
+
05/12/2026 04:51:35 - INFO - root - epochs: 2/2, steps: 1950/3872, func: 0.051019, pos: 0.057938, neg: 0.028023, 50%: 1h 49m 55s
|
| 54 |
+
05/12/2026 04:54:29 - INFO - root - epochs: 2/2, steps: 2000/3872, func: 0.051679, pos: 0.063034, neg: 0.015457, 51%: 1h 47m 7s
|
| 55 |
+
05/12/2026 04:57:21 - INFO - root - epochs: 2/2, steps: 2050/3872, func: 0.052008, pos: 0.057971, neg: 0.025081, 52%: 1h 44m 15s
|
| 56 |
+
05/12/2026 05:00:15 - INFO - root - epochs: 2/2, steps: 2100/3872, func: 0.050413, pos: 0.057375, neg: 0.020022, 54%: 1h 41m 26s
|
| 57 |
+
05/12/2026 05:03:05 - INFO - root - epochs: 2/2, steps: 2150/3872, func: 0.051819, pos: 0.061493, neg: 0.02545, 55%: 1h 38m 33s
|
| 58 |
+
05/12/2026 05:05:58 - INFO - root - epochs: 2/2, steps: 2200/3872, func: 0.051764, pos: 0.05972, neg: 0.026174, 56%: 1h 35m 42s
|
| 59 |
+
05/12/2026 05:08:44 - INFO - root - epochs: 2/2, steps: 2250/3872, func: 0.050719, pos: 0.053116, neg: 0.01747, 58%: 1h 32m 47s
|
| 60 |
+
05/12/2026 05:11:36 - INFO - root - epochs: 2/2, steps: 2300/3872, func: 0.050605, pos: 0.036939, neg: 0.015887, 59%: 1h 29m 56s
|
| 61 |
+
05/12/2026 05:14:28 - INFO - root - epochs: 2/2, steps: 2350/3872, func: 0.051366, pos: 0.044932, neg: 0.028288, 60%: 1h 27m 4s
|
| 62 |
+
05/12/2026 05:17:17 - INFO - root - epochs: 2/2, steps: 2400/3872, func: 0.05086, pos: 0.056528, neg: 0.019473, 61%: 1h 24m 12s
|
| 63 |
+
05/12/2026 05:20:13 - INFO - root - epochs: 2/2, steps: 2450/3872, func: 0.050416, pos: 0.056659, neg: 0.026592, 63%: 1h 21m 23s
|
| 64 |
+
05/12/2026 05:23:05 - INFO - root - epochs: 2/2, steps: 2500/3872, func: 0.051684, pos: 0.070944, neg: 0.023598, 64%: 1h 18m 31s
|
| 65 |
+
05/12/2026 05:25:55 - INFO - root - epochs: 2/2, steps: 2550/3872, func: 0.051356, pos: 0.054086, neg: 0.021948, 65%: 1h 15m 39s
|
| 66 |
+
05/12/2026 05:28:47 - INFO - root - epochs: 2/2, steps: 2600/3872, func: 0.050023, pos: 0.045485, neg: 0.024609, 67%: 1h 12m 48s
|
| 67 |
+
05/12/2026 05:31:37 - INFO - root - epochs: 2/2, steps: 2650/3872, func: 0.05107, pos: 0.038352, neg: 0.025448, 68%: 1h 9m 55s
|
| 68 |
+
05/12/2026 05:34:29 - INFO - root - epochs: 2/2, steps: 2700/3872, func: 0.05065, pos: 0.076865, neg: 0.02363, 69%: 1h 7m 4s
|
| 69 |
+
05/12/2026 05:37:23 - INFO - root - epochs: 2/2, steps: 2750/3872, func: 0.051486, pos: 0.040869, neg: 0.013997, 70%: 1h 4m 13s
|
| 70 |
+
05/12/2026 05:40:16 - INFO - root - epochs: 2/2, steps: 2800/3872, func: 0.051778, pos: 0.053576, neg: 0.019081, 72%: 1h 1m 22s
|
| 71 |
+
05/12/2026 05:43:09 - INFO - root - epochs: 2/2, steps: 2850/3872, func: 0.050467, pos: 0.064041, neg: 0.024908, 73%: 0h 58m 31s
|
| 72 |
+
05/12/2026 05:46:02 - INFO - root - epochs: 2/2, steps: 2900/3872, func: 0.050543, pos: 0.053341, neg: 0.026226, 74%: 0h 55m 40s
|
| 73 |
+
05/12/2026 05:48:58 - INFO - root - epochs: 2/2, steps: 2950/3872, func: 0.051039, pos: 0.054438, neg: 0.026954, 76%: 0h 52m 50s
|
| 74 |
+
05/12/2026 05:51:48 - INFO - root - epochs: 2/2, steps: 3000/3872, func: 0.050754, pos: 0.049894, neg: 0.018876, 77%: 0h 49m 57s
|
| 75 |
+
05/12/2026 05:54:38 - INFO - root - epochs: 2/2, steps: 3050/3872, func: 0.050262, pos: 0.067919, neg: 0.024587, 78%: 0h 47m 5s
|
| 76 |
+
05/12/2026 05:57:28 - INFO - root - epochs: 2/2, steps: 3100/3872, func: 0.050994, pos: 0.065647, neg: 0.015296, 80%: 0h 44m 13s
|
| 77 |
+
05/12/2026 06:00:19 - INFO - root - epochs: 2/2, steps: 3150/3872, func: 0.051998, pos: 0.0547, neg: 0.019967, 81%: 0h 41m 21s
|
| 78 |
+
05/12/2026 06:03:10 - INFO - root - epochs: 2/2, steps: 3200/3872, func: 0.050904, pos: 0.051526, neg: 0.014749, 82%: 0h 38m 30s
|
| 79 |
+
05/12/2026 06:05:58 - INFO - root - epochs: 2/2, steps: 3250/3872, func: 0.051783, pos: 0.051603, neg: 0.020233, 83%: 0h 35m 37s
|
| 80 |
+
05/12/2026 06:08:50 - INFO - root - epochs: 2/2, steps: 3300/3872, func: 0.052165, pos: 0.068071, neg: 0.023161, 85%: 0h 32m 46s
|
| 81 |
+
05/12/2026 06:11:41 - INFO - root - epochs: 2/2, steps: 3350/3872, func: 0.050292, pos: 0.044843, neg: 0.018546, 86%: 0h 29m 54s
|
| 82 |
+
05/12/2026 06:14:31 - INFO - root - epochs: 2/2, steps: 3400/3872, func: 0.050373, pos: 0.035594, neg: 0.023604, 87%: 0h 27m 2s
|
| 83 |
+
05/12/2026 06:17:19 - INFO - root - epochs: 2/2, steps: 3450/3872, func: 0.051031, pos: 0.051028, neg: 0.022269, 89%: 0h 24m 10s
|
| 84 |
+
05/12/2026 06:20:10 - INFO - root - epochs: 2/2, steps: 3500/3872, func: 0.050252, pos: 0.049759, neg: 0.018131, 90%: 0h 21m 19s
|
| 85 |
+
05/12/2026 06:23:02 - INFO - root - epochs: 2/2, steps: 3550/3872, func: 0.05084, pos: 0.039536, neg: 0.027638, 91%: 0h 18m 27s
|
| 86 |
+
05/12/2026 06:25:50 - INFO - root - epochs: 2/2, steps: 3600/3872, func: 0.05144, pos: 0.048153, neg: 0.024605, 92%: 0h 15m 36s
|
| 87 |
+
05/12/2026 06:28:41 - INFO - root - epochs: 2/2, steps: 3650/3872, func: 0.05042, pos: 0.045741, neg: 0.017756, 94%: 0h 12m 44s
|
| 88 |
+
05/12/2026 06:31:33 - INFO - root - epochs: 2/2, steps: 3700/3872, func: 0.051448, pos: 0.036074, neg: 0.013317, 95%: 0h 9m 53s
|
| 89 |
+
05/12/2026 06:34:25 - INFO - root - epochs: 2/2, steps: 3750/3872, func: 0.050891, pos: 0.063433, neg: 0.028583, 96%: 0h 7m 1s
|
| 90 |
+
05/12/2026 06:37:18 - INFO - root - epochs: 2/2, steps: 3800/3872, func: 0.051766, pos: 0.056375, neg: 0.030348, 98%: 0h 4m 10s
|
| 91 |
+
05/12/2026 06:40:11 - INFO - root - epochs: 2/2, steps: 3850/3872, func: 0.051351, pos: 0.049689, neg: 0.018709, 99%: 0h 1m 18s
|
| 92 |
+
05/12/2026 06:47:02 - INFO - root - final eval loss: func: 0.051307, pos: 0.062347, neg: 0.029495
|
| 93 |
+
05/12/2026 06:47:02 - INFO - root - Saving model checkpoint to ../trained/deepseek-coder-1.3b-lora-safecoder/checkpoint-last
|
deepseek-coder-6.7b-lora-safecoder/checkpoint-last/README.md
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: deepseek-ai/deepseek-coder-6.7b-base
|
| 3 |
+
library_name: peft
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Model Card for Model ID
|
| 7 |
+
|
| 8 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
## Model Details
|
| 13 |
+
|
| 14 |
+
### Model Description
|
| 15 |
+
|
| 16 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
- **Developed by:** [More Information Needed]
|
| 21 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 22 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 23 |
+
- **Model type:** [More Information Needed]
|
| 24 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 25 |
+
- **License:** [More Information Needed]
|
| 26 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 27 |
+
|
| 28 |
+
### Model Sources [optional]
|
| 29 |
+
|
| 30 |
+
<!-- Provide the basic links for the model. -->
|
| 31 |
+
|
| 32 |
+
- **Repository:** [More Information Needed]
|
| 33 |
+
- **Paper [optional]:** [More Information Needed]
|
| 34 |
+
- **Demo [optional]:** [More Information Needed]
|
| 35 |
+
|
| 36 |
+
## Uses
|
| 37 |
+
|
| 38 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 39 |
+
|
| 40 |
+
### Direct Use
|
| 41 |
+
|
| 42 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 43 |
+
|
| 44 |
+
[More Information Needed]
|
| 45 |
+
|
| 46 |
+
### Downstream Use [optional]
|
| 47 |
+
|
| 48 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 49 |
+
|
| 50 |
+
[More Information Needed]
|
| 51 |
+
|
| 52 |
+
### Out-of-Scope Use
|
| 53 |
+
|
| 54 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 55 |
+
|
| 56 |
+
[More Information Needed]
|
| 57 |
+
|
| 58 |
+
## Bias, Risks, and Limitations
|
| 59 |
+
|
| 60 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 61 |
+
|
| 62 |
+
[More Information Needed]
|
| 63 |
+
|
| 64 |
+
### Recommendations
|
| 65 |
+
|
| 66 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 67 |
+
|
| 68 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 69 |
+
|
| 70 |
+
## How to Get Started with the Model
|
| 71 |
+
|
| 72 |
+
Use the code below to get started with the model.
|
| 73 |
+
|
| 74 |
+
[More Information Needed]
|
| 75 |
+
|
| 76 |
+
## Training Details
|
| 77 |
+
|
| 78 |
+
### Training Data
|
| 79 |
+
|
| 80 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 81 |
+
|
| 82 |
+
[More Information Needed]
|
| 83 |
+
|
| 84 |
+
### Training Procedure
|
| 85 |
+
|
| 86 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 87 |
+
|
| 88 |
+
#### Preprocessing [optional]
|
| 89 |
+
|
| 90 |
+
[More Information Needed]
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
#### Training Hyperparameters
|
| 94 |
+
|
| 95 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 96 |
+
|
| 97 |
+
#### Speeds, Sizes, Times [optional]
|
| 98 |
+
|
| 99 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 100 |
+
|
| 101 |
+
[More Information Needed]
|
| 102 |
+
|
| 103 |
+
## Evaluation
|
| 104 |
+
|
| 105 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 106 |
+
|
| 107 |
+
### Testing Data, Factors & Metrics
|
| 108 |
+
|
| 109 |
+
#### Testing Data
|
| 110 |
+
|
| 111 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 112 |
+
|
| 113 |
+
[More Information Needed]
|
| 114 |
+
|
| 115 |
+
#### Factors
|
| 116 |
+
|
| 117 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 118 |
+
|
| 119 |
+
[More Information Needed]
|
| 120 |
+
|
| 121 |
+
#### Metrics
|
| 122 |
+
|
| 123 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 124 |
+
|
| 125 |
+
[More Information Needed]
|
| 126 |
+
|
| 127 |
+
### Results
|
| 128 |
+
|
| 129 |
+
[More Information Needed]
|
| 130 |
+
|
| 131 |
+
#### Summary
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
## Model Examination [optional]
|
| 136 |
+
|
| 137 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 138 |
+
|
| 139 |
+
[More Information Needed]
|
| 140 |
+
|
| 141 |
+
## Environmental Impact
|
| 142 |
+
|
| 143 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 144 |
+
|
| 145 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 146 |
+
|
| 147 |
+
- **Hardware Type:** [More Information Needed]
|
| 148 |
+
- **Hours used:** [More Information Needed]
|
| 149 |
+
- **Cloud Provider:** [More Information Needed]
|
| 150 |
+
- **Compute Region:** [More Information Needed]
|
| 151 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 152 |
+
|
| 153 |
+
## Technical Specifications [optional]
|
| 154 |
+
|
| 155 |
+
### Model Architecture and Objective
|
| 156 |
+
|
| 157 |
+
[More Information Needed]
|
| 158 |
+
|
| 159 |
+
### Compute Infrastructure
|
| 160 |
+
|
| 161 |
+
[More Information Needed]
|
| 162 |
+
|
| 163 |
+
#### Hardware
|
| 164 |
+
|
| 165 |
+
[More Information Needed]
|
| 166 |
+
|
| 167 |
+
#### Software
|
| 168 |
+
|
| 169 |
+
[More Information Needed]
|
| 170 |
+
|
| 171 |
+
## Citation [optional]
|
| 172 |
+
|
| 173 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 174 |
+
|
| 175 |
+
**BibTeX:**
|
| 176 |
+
|
| 177 |
+
[More Information Needed]
|
| 178 |
+
|
| 179 |
+
**APA:**
|
| 180 |
+
|
| 181 |
+
[More Information Needed]
|
| 182 |
+
|
| 183 |
+
## Glossary [optional]
|
| 184 |
+
|
| 185 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 186 |
+
|
| 187 |
+
[More Information Needed]
|
| 188 |
+
|
| 189 |
+
## More Information [optional]
|
| 190 |
+
|
| 191 |
+
[More Information Needed]
|
| 192 |
+
|
| 193 |
+
## Model Card Authors [optional]
|
| 194 |
+
|
| 195 |
+
[More Information Needed]
|
| 196 |
+
|
| 197 |
+
## Model Card Contact
|
| 198 |
+
|
| 199 |
+
[More Information Needed]
|
| 200 |
+
### Framework versions
|
| 201 |
+
|
| 202 |
+
- PEFT 0.8.2
|
deepseek-coder-6.7b-lora-safecoder/checkpoint-last/adapter_config.json
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alpha_pattern": {},
|
| 3 |
+
"auto_mapping": null,
|
| 4 |
+
"base_model_name_or_path": "deepseek-ai/deepseek-coder-6.7b-base",
|
| 5 |
+
"bias": "none",
|
| 6 |
+
"fan_in_fan_out": false,
|
| 7 |
+
"inference_mode": true,
|
| 8 |
+
"init_lora_weights": true,
|
| 9 |
+
"layers_pattern": null,
|
| 10 |
+
"layers_to_transform": null,
|
| 11 |
+
"loftq_config": {},
|
| 12 |
+
"lora_alpha": 32,
|
| 13 |
+
"lora_dropout": 0.1,
|
| 14 |
+
"megatron_config": null,
|
| 15 |
+
"megatron_core": "megatron.core",
|
| 16 |
+
"modules_to_save": null,
|
| 17 |
+
"peft_type": "LORA",
|
| 18 |
+
"r": 16,
|
| 19 |
+
"rank_pattern": {},
|
| 20 |
+
"revision": null,
|
| 21 |
+
"target_modules": [
|
| 22 |
+
"v_proj",
|
| 23 |
+
"gate_proj",
|
| 24 |
+
"o_proj",
|
| 25 |
+
"up_proj",
|
| 26 |
+
"q_proj",
|
| 27 |
+
"k_proj",
|
| 28 |
+
"lm_head",
|
| 29 |
+
"down_proj"
|
| 30 |
+
],
|
| 31 |
+
"task_type": "CAUSAL_LM",
|
| 32 |
+
"use_rslora": false
|
| 33 |
+
}
|
deepseek-coder-6.7b-lora-safecoder/checkpoint-last/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ed0c4694e5180dd1e6f24bb3b67fa7d0704f45db5575022a73c259a6a0c8b691
|
| 3 |
+
size 686928848
|
deepseek-coder-6.7b-lora-safecoder/checkpoint-last/special_tokens_map.json
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token": {
|
| 3 |
+
"content": "<｜begin▁of▁sentence｜>",
|
| 4 |
+
"lstrip": false,
|
| 5 |
+
"normalized": true,
|
| 6 |
+
"rstrip": false,
|
| 7 |
+
"single_word": false
|
| 8 |
+
},
|
| 9 |
+
"eos_token": {
|
| 10 |
+
"content": "<｜end▁of▁sentence｜>",
|
| 11 |
+
"lstrip": false,
|
| 12 |
+
"normalized": true,
|
| 13 |
+
"rstrip": false,
|
| 14 |
+
"single_word": false
|
| 15 |
+
},
|
| 16 |
+
"pad_token": {
|
| 17 |
+
"content": "<｜end▁of▁sentence｜>",
|
| 18 |
+
"lstrip": false,
|
| 19 |
+
"normalized": true,
|
| 20 |
+
"rstrip": false,
|
| 21 |
+
"single_word": false
|
| 22 |
+
}
|
| 23 |
+
}
|
deepseek-coder-6.7b-lora-safecoder/checkpoint-last/tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
deepseek-coder-6.7b-lora-safecoder/checkpoint-last/tokenizer_config.json
ADDED
|
@@ -0,0 +1,194 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_bos_token": true,
|
| 3 |
+
"add_eos_token": false,
|
| 4 |
+
"add_prefix_space": null,
|
| 5 |
+
"added_tokens_decoder": {
|
| 6 |
+
"32000": {
|
| 7 |
+
"content": "õ",
|
| 8 |
+
"lstrip": false,
|
| 9 |
+
"normalized": true,
|
| 10 |
+
"rstrip": false,
|
| 11 |
+
"single_word": false,
|
| 12 |
+
"special": false
|
| 13 |
+
},
|
| 14 |
+
"32001": {
|
| 15 |
+
"content": "÷",
|
| 16 |
+
"lstrip": false,
|
| 17 |
+
"normalized": true,
|
| 18 |
+
"rstrip": false,
|
| 19 |
+
"single_word": false,
|
| 20 |
+
"special": false
|
| 21 |
+
},
|
| 22 |
+
"32002": {
|
| 23 |
+
"content": "Á",
|
| 24 |
+
"lstrip": false,
|
| 25 |
+
"normalized": true,
|
| 26 |
+
"rstrip": false,
|
| 27 |
+
"single_word": false,
|
| 28 |
+
"special": false
|
| 29 |
+
},
|
| 30 |
+
"32003": {
|
| 31 |
+
"content": "ý",
|
| 32 |
+
"lstrip": false,
|
| 33 |
+
"normalized": true,
|
| 34 |
+
"rstrip": false,
|
| 35 |
+
"single_word": false,
|
| 36 |
+
"special": false
|
| 37 |
+
},
|
| 38 |
+
"32004": {
|
| 39 |
+
"content": "À",
|
| 40 |
+
"lstrip": false,
|
| 41 |
+
"normalized": true,
|
| 42 |
+
"rstrip": false,
|
| 43 |
+
"single_word": false,
|
| 44 |
+
"special": false
|
| 45 |
+
},
|
| 46 |
+
"32005": {
|
| 47 |
+
"content": "ÿ",
|
| 48 |
+
"lstrip": false,
|
| 49 |
+
"normalized": true,
|
| 50 |
+
"rstrip": false,
|
| 51 |
+
"single_word": false,
|
| 52 |
+
"special": false
|
| 53 |
+
},
|
| 54 |
+
"32006": {
|
| 55 |
+
"content": "ø",
|
| 56 |
+
"lstrip": false,
|
| 57 |
+
"normalized": true,
|
| 58 |
+
"rstrip": false,
|
| 59 |
+
"single_word": false,
|
| 60 |
+
"special": false
|
| 61 |
+
},
|
| 62 |
+
"32007": {
|
| 63 |
+
"content": "ú",
|
| 64 |
+
"lstrip": false,
|
| 65 |
+
"normalized": true,
|
| 66 |
+
"rstrip": false,
|
| 67 |
+
"single_word": false,
|
| 68 |
+
"special": false
|
| 69 |
+
},
|
| 70 |
+
"32008": {
|
| 71 |
+
"content": "þ",
|
| 72 |
+
"lstrip": false,
|
| 73 |
+
"normalized": true,
|
| 74 |
+
"rstrip": false,
|
| 75 |
+
"single_word": false,
|
| 76 |
+
"special": false
|
| 77 |
+
},
|
| 78 |
+
"32009": {
|
| 79 |
+
"content": "ü",
|
| 80 |
+
"lstrip": false,
|
| 81 |
+
"normalized": true,
|
| 82 |
+
"rstrip": false,
|
| 83 |
+
"single_word": false,
|
| 84 |
+
"special": false
|
| 85 |
+
},
|
| 86 |
+
"32010": {
|
| 87 |
+
"content": "ù",
|
| 88 |
+
"lstrip": false,
|
| 89 |
+
"normalized": true,
|
| 90 |
+
"rstrip": false,
|
| 91 |
+
"single_word": false,
|
| 92 |
+
"special": false
|
| 93 |
+
},
|
| 94 |
+
"32011": {
|
| 95 |
+
"content": "ö",
|
| 96 |
+
"lstrip": false,
|
| 97 |
+
"normalized": true,
|
| 98 |
+
"rstrip": false,
|
| 99 |
+
"single_word": false,
|
| 100 |
+
"special": false
|
| 101 |
+
},
|
| 102 |
+
"32012": {
|
| 103 |
+
"content": "û",
|
| 104 |
+
"lstrip": false,
|
| 105 |
+
"normalized": true,
|
| 106 |
+
"rstrip": false,
|
| 107 |
+
"single_word": false,
|
| 108 |
+
"special": false
|
| 109 |
+
},
|
| 110 |
+
"32013": {
|
| 111 |
+
"content": "<｜begin▁of▁sentence｜>",
|
| 112 |
+
"lstrip": false,
|
| 113 |
+
"normalized": true,
|
| 114 |
+
"rstrip": false,
|
| 115 |
+
"single_word": false,
|
| 116 |
+
"special": true
|
| 117 |
+
},
|
| 118 |
+
"32014": {
|
| 119 |
+
"content": "<｜end▁of▁sentence｜>",
|
| 120 |
+
"lstrip": false,
|
| 121 |
+
"normalized": true,
|
| 122 |
+
"rstrip": false,
|
| 123 |
+
"single_word": false,
|
| 124 |
+
"special": true
|
| 125 |
+
},
|
| 126 |
+
"32015": {
|
| 127 |
+
"content": "<｜fim▁hole｜>",
|
| 128 |
+
"lstrip": false,
|
| 129 |
+
"normalized": true,
|
| 130 |
+
"rstrip": false,
|
| 131 |
+
"single_word": false,
|
| 132 |
+
"special": false
|
| 133 |
+
},
|
| 134 |
+
"32016": {
|
| 135 |
+
"content": "<｜fim▁begin｜>",
|
| 136 |
+
"lstrip": false,
|
| 137 |
+
"normalized": true,
|
| 138 |
+
"rstrip": false,
|
| 139 |
+
"single_word": false,
|
| 140 |
+
"special": false
|
| 141 |
+
},
|
| 142 |
+
"32017": {
|
| 143 |
+
"content": "<｜fim▁end｜>",
|
| 144 |
+
"lstrip": false,
|
| 145 |
+
"normalized": true,
|
| 146 |
+
"rstrip": false,
|
| 147 |
+
"single_word": false,
|
| 148 |
+
"special": false
|
| 149 |
+
},
|
| 150 |
+
"32018": {
|
| 151 |
+
"content": "<pad>",
|
| 152 |
+
"lstrip": false,
|
| 153 |
+
"normalized": true,
|
| 154 |
+
"rstrip": false,
|
| 155 |
+
"single_word": false,
|
| 156 |
+
"special": false
|
| 157 |
+
},
|
| 158 |
+
"32019": {
|
| 159 |
+
"content": "<|User|>",
|
| 160 |
+
"lstrip": false,
|
| 161 |
+
"normalized": true,
|
| 162 |
+
"rstrip": false,
|
| 163 |
+
"single_word": false,
|
| 164 |
+
"special": false
|
| 165 |
+
},
|
| 166 |
+
"32020": {
|
| 167 |
+
"content": "<|Assistant|>",
|
| 168 |
+
"lstrip": false,
|
| 169 |
+
"normalized": true,
|
| 170 |
+
"rstrip": false,
|
| 171 |
+
"single_word": false,
|
| 172 |
+
"special": false
|
| 173 |
+
},
|
| 174 |
+
"32021": {
|
| 175 |
+
"content": "<|EOT|>",
|
| 176 |
+
"lstrip": false,
|
| 177 |
+
"normalized": true,
|
| 178 |
+
"rstrip": false,
|
| 179 |
+
"single_word": false,
|
| 180 |
+
"special": false
|
| 181 |
+
}
|
| 182 |
+
},
|
| 183 |
+
"bos_token": "<｜begin▁of▁sentence｜>",
|
| 184 |
+
"clean_up_tokenization_spaces": false,
|
| 185 |
+
"eos_token": "<｜end▁of▁sentence｜>",
|
| 186 |
+
"extra_special_tokens": {},
|
| 187 |
+
"legacy": true,
|
| 188 |
+
"model_max_length": 16384,
|
| 189 |
+
"pad_token": "<｜end▁of▁sentence｜>",
|
| 190 |
+
"sp_model_kwargs": {},
|
| 191 |
+
"tokenizer_class": "LlamaTokenizerFast",
|
| 192 |
+
"unk_token": null,
|
| 193 |
+
"use_default_system_prompt": false
|
| 194 |
+
}
|
deepseek-coder-6.7b-lora-safecoder/train.log
ADDED
|
Binary file (13 kB). View file
|
|
|
llama2-7b-lora-safecoder/checkpoint-last/README.md
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: meta-llama/Llama-2-7b-hf
|
| 3 |
+
library_name: peft
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Model Card for Model ID
|
| 7 |
+
|
| 8 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
## Model Details
|
| 13 |
+
|
| 14 |
+
### Model Description
|
| 15 |
+
|
| 16 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
- **Developed by:** [More Information Needed]
|
| 21 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 22 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 23 |
+
- **Model type:** [More Information Needed]
|
| 24 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 25 |
+
- **License:** [More Information Needed]
|
| 26 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 27 |
+
|
| 28 |
+
### Model Sources [optional]
|
| 29 |
+
|
| 30 |
+
<!-- Provide the basic links for the model. -->
|
| 31 |
+
|
| 32 |
+
- **Repository:** [More Information Needed]
|
| 33 |
+
- **Paper [optional]:** [More Information Needed]
|
| 34 |
+
- **Demo [optional]:** [More Information Needed]
|
| 35 |
+
|
| 36 |
+
## Uses
|
| 37 |
+
|
| 38 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 39 |
+
|
| 40 |
+
### Direct Use
|
| 41 |
+
|
| 42 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 43 |
+
|
| 44 |
+
[More Information Needed]
|
| 45 |
+
|
| 46 |
+
### Downstream Use [optional]
|
| 47 |
+
|
| 48 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 49 |
+
|
| 50 |
+
[More Information Needed]
|
| 51 |
+
|
| 52 |
+
### Out-of-Scope Use
|
| 53 |
+
|
| 54 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 55 |
+
|
| 56 |
+
[More Information Needed]
|
| 57 |
+
|
| 58 |
+
## Bias, Risks, and Limitations
|
| 59 |
+
|
| 60 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 61 |
+
|
| 62 |
+
[More Information Needed]
|
| 63 |
+
|
| 64 |
+
### Recommendations
|
| 65 |
+
|
| 66 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 67 |
+
|
| 68 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 69 |
+
|
| 70 |
+
## How to Get Started with the Model
|
| 71 |
+
|
| 72 |
+
Use the code below to get started with the model.
|
| 73 |
+
|
| 74 |
+
[More Information Needed]
|
| 75 |
+
|
| 76 |
+
## Training Details
|
| 77 |
+
|
| 78 |
+
### Training Data
|
| 79 |
+
|
| 80 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 81 |
+
|
| 82 |
+
[More Information Needed]
|
| 83 |
+
|
| 84 |
+
### Training Procedure
|
| 85 |
+
|
| 86 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 87 |
+
|
| 88 |
+
#### Preprocessing [optional]
|
| 89 |
+
|
| 90 |
+
[More Information Needed]
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
#### Training Hyperparameters
|
| 94 |
+
|
| 95 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 96 |
+
|
| 97 |
+
#### Speeds, Sizes, Times [optional]
|
| 98 |
+
|
| 99 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 100 |
+
|
| 101 |
+
[More Information Needed]
|
| 102 |
+
|
| 103 |
+
## Evaluation
|
| 104 |
+
|
| 105 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 106 |
+
|
| 107 |
+
### Testing Data, Factors & Metrics
|
| 108 |
+
|
| 109 |
+
#### Testing Data
|
| 110 |
+
|
| 111 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 112 |
+
|
| 113 |
+
[More Information Needed]
|
| 114 |
+
|
| 115 |
+
#### Factors
|
| 116 |
+
|
| 117 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 118 |
+
|
| 119 |
+
[More Information Needed]
|
| 120 |
+
|
| 121 |
+
#### Metrics
|
| 122 |
+
|
| 123 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 124 |
+
|
| 125 |
+
[More Information Needed]
|
| 126 |
+
|
| 127 |
+
### Results
|
| 128 |
+
|
| 129 |
+
[More Information Needed]
|
| 130 |
+
|
| 131 |
+
#### Summary
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
## Model Examination [optional]
|
| 136 |
+
|
| 137 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 138 |
+
|
| 139 |
+
[More Information Needed]
|
| 140 |
+
|
| 141 |
+
## Environmental Impact
|
| 142 |
+
|
| 143 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 144 |
+
|
| 145 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 146 |
+
|
| 147 |
+
- **Hardware Type:** [More Information Needed]
|
| 148 |
+
- **Hours used:** [More Information Needed]
|
| 149 |
+
- **Cloud Provider:** [More Information Needed]
|
| 150 |
+
- **Compute Region:** [More Information Needed]
|
| 151 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 152 |
+
|
| 153 |
+
## Technical Specifications [optional]
|
| 154 |
+
|
| 155 |
+
### Model Architecture and Objective
|
| 156 |
+
|
| 157 |
+
[More Information Needed]
|
| 158 |
+
|
| 159 |
+
### Compute Infrastructure
|
| 160 |
+
|
| 161 |
+
[More Information Needed]
|
| 162 |
+
|
| 163 |
+
#### Hardware
|
| 164 |
+
|
| 165 |
+
[More Information Needed]
|
| 166 |
+
|
| 167 |
+
#### Software
|
| 168 |
+
|
| 169 |
+
[More Information Needed]
|
| 170 |
+
|
| 171 |
+
## Citation [optional]
|
| 172 |
+
|
| 173 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 174 |
+
|
| 175 |
+
**BibTeX:**
|
| 176 |
+
|
| 177 |
+
[More Information Needed]
|
| 178 |
+
|
| 179 |
+
**APA:**
|
| 180 |
+
|
| 181 |
+
[More Information Needed]
|
| 182 |
+
|
| 183 |
+
## Glossary [optional]
|
| 184 |
+
|
| 185 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 186 |
+
|
| 187 |
+
[More Information Needed]
|
| 188 |
+
|
| 189 |
+
## More Information [optional]
|
| 190 |
+
|
| 191 |
+
[More Information Needed]
|
| 192 |
+
|
| 193 |
+
## Model Card Authors [optional]
|
| 194 |
+
|
| 195 |
+
[More Information Needed]
|
| 196 |
+
|
| 197 |
+
## Model Card Contact
|
| 198 |
+
|
| 199 |
+
[More Information Needed]
|
| 200 |
+
### Framework versions
|
| 201 |
+
|
| 202 |
+
- PEFT 0.8.2
|
llama2-7b-lora-safecoder/checkpoint-last/adapter_config.json
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alpha_pattern": {},
|
| 3 |
+
"auto_mapping": null,
|
| 4 |
+
"base_model_name_or_path": "meta-llama/Llama-2-7b-hf",
|
| 5 |
+
"bias": "none",
|
| 6 |
+
"fan_in_fan_out": false,
|
| 7 |
+
"inference_mode": true,
|
| 8 |
+
"init_lora_weights": true,
|
| 9 |
+
"layers_pattern": null,
|
| 10 |
+
"layers_to_transform": null,
|
| 11 |
+
"loftq_config": {},
|
| 12 |
+
"lora_alpha": 32,
|
| 13 |
+
"lora_dropout": 0.1,
|
| 14 |
+
"megatron_config": null,
|
| 15 |
+
"megatron_core": "megatron.core",
|
| 16 |
+
"modules_to_save": null,
|
| 17 |
+
"peft_type": "LORA",
|
| 18 |
+
"r": 16,
|
| 19 |
+
"rank_pattern": {},
|
| 20 |
+
"revision": null,
|
| 21 |
+
"target_modules": [
|
| 22 |
+
"gate_proj",
|
| 23 |
+
"v_proj",
|
| 24 |
+
"down_proj",
|
| 25 |
+
"up_proj",
|
| 26 |
+
"o_proj",
|
| 27 |
+
"k_proj",
|
| 28 |
+
"lm_head",
|
| 29 |
+
"q_proj"
|
| 30 |
+
],
|
| 31 |
+
"task_type": "CAUSAL_LM",
|
| 32 |
+
"use_rslora": false
|
| 33 |
+
}
|
llama2-7b-lora-safecoder/checkpoint-last/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8d1cb5510a82b8fa11c757663e73195934f2159832cf9c80e986aeae77767c25
|
| 3 |
+
size 686566992
|
llama2-7b-lora-safecoder/checkpoint-last/special_tokens_map.json
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token": {
|
| 3 |
+
"content": "<s>",
|
| 4 |
+
"lstrip": false,
|
| 5 |
+
"normalized": false,
|
| 6 |
+
"rstrip": false,
|
| 7 |
+
"single_word": false
|
| 8 |
+
},
|
| 9 |
+
"eos_token": {
|
| 10 |
+
"content": "</s>",
|
| 11 |
+
"lstrip": false,
|
| 12 |
+
"normalized": false,
|
| 13 |
+
"rstrip": false,
|
| 14 |
+
"single_word": false
|
| 15 |
+
},
|
| 16 |
+
"unk_token": {
|
| 17 |
+
"content": "<unk>",
|
| 18 |
+
"lstrip": false,
|
| 19 |
+
"normalized": false,
|
| 20 |
+
"rstrip": false,
|
| 21 |
+
"single_word": false
|
| 22 |
+
}
|
| 23 |
+
}
|
llama2-7b-lora-safecoder/checkpoint-last/tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
llama2-7b-lora-safecoder/checkpoint-last/tokenizer.model
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
|
| 3 |
+
size 499723
|
llama2-7b-lora-safecoder/checkpoint-last/tokenizer_config.json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_bos_token": true,
|
| 3 |
+
"add_eos_token": false,
|
| 4 |
+
"add_prefix_space": null,
|
| 5 |
+
"added_tokens_decoder": {
|
| 6 |
+
"0": {
|
| 7 |
+
"content": "<unk>",
|
| 8 |
+
"lstrip": false,
|
| 9 |
+
"normalized": false,
|
| 10 |
+
"rstrip": false,
|
| 11 |
+
"single_word": false,
|
| 12 |
+
"special": true
|
| 13 |
+
},
|
| 14 |
+
"1": {
|
| 15 |
+
"content": "<s>",
|
| 16 |
+
"lstrip": false,
|
| 17 |
+
"normalized": false,
|
| 18 |
+
"rstrip": false,
|
| 19 |
+
"single_word": false,
|
| 20 |
+
"special": true
|
| 21 |
+
},
|
| 22 |
+
"2": {
|
| 23 |
+
"content": "</s>",
|
| 24 |
+
"lstrip": false,
|
| 25 |
+
"normalized": false,
|
| 26 |
+
"rstrip": false,
|
| 27 |
+
"single_word": false,
|
| 28 |
+
"special": true
|
| 29 |
+
}
|
| 30 |
+
},
|
| 31 |
+
"bos_token": "<s>",
|
| 32 |
+
"clean_up_tokenization_spaces": false,
|
| 33 |
+
"eos_token": "</s>",
|
| 34 |
+
"extra_special_tokens": {},
|
| 35 |
+
"legacy": false,
|
| 36 |
+
"model_max_length": 1000000000000000019884624838656,
|
| 37 |
+
"pad_token": null,
|
| 38 |
+
"padding_side": "right",
|
| 39 |
+
"sp_model_kwargs": {},
|
| 40 |
+
"tokenizer_class": "LlamaTokenizer",
|
| 41 |
+
"unk_token": "<unk>",
|
| 42 |
+
"use_default_system_prompt": false
|
| 43 |
+
}
|
llama2-7b-lora-safecoder/train.log
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
05/12/2026 04:15:26 - INFO - accelerate.utils.modeling - We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can set `max_memory` in to a higher value to use more memory (at your own risk).
|
| 2 |
+
05/12/2026 04:16:11 - INFO - root - number of sec samples before upsampling: 1806
|
| 3 |
+
05/12/2026 04:16:11 - INFO - root - number of sec samples after upsampling: 3305
|
| 4 |
+
05/12/2026 04:16:13 - INFO - root - Training args Namespace(output_name='llama2-7b-lora-safecoder', datasets=['lmsys', 'sec-desc', 'sec-new-desc'], pretrain_name='llama2-7b', loss_weight=1.0, sven=False, num_train_epochs=2, learning_rate=2e-05, max_num_tokens=1024, batch_size=1, grad_acc_steps=16, weight_decay=0.01, adam_epsilon=1e-08, warmup_steps=0, max_grad_norm=1.0, dropout=0.1, kl_loss_weight=0, exclude_neg=False, no_weights=False, lora=True, r=16, lora_alpha=32, lora_dropout=0.1, sampling_size=40, sampling_method='minority', cwes=['all'], langs=['all'], logging_steps=50, save_epochs=10, seed=2, data_dir='../data_train_val', model_dir='../trained/', output_dir='../trained/llama2-7b-lora-safecoder', logger=<RootLogger root (INFO)>)
|
| 5 |
+
05/12/2026 04:16:13 - INFO - root - ***** Running training *****
|
| 6 |
+
05/12/2026 04:16:13 - INFO - root - Num samples = 19499
|
| 7 |
+
05/12/2026 04:16:13 - INFO - root - Num epoch = 2
|
| 8 |
+
05/12/2026 04:16:13 - INFO - root - Batch size= 1
|
| 9 |
+
05/12/2026 04:16:13 - INFO - root - Total batch size (w. accumulation) = 16
|
| 10 |
+
05/12/2026 04:16:13 - INFO - root - Gradient Accumulation steps = 16
|
| 11 |
+
05/12/2026 04:16:13 - INFO - root - Total optimization steps = 2436
|
| 12 |
+
05/12/2026 04:16:13 - INFO - root - Num val samples = 1994
|
| 13 |
+
05/12/2026 04:16:13 - INFO - root - Num parameters = 6778970112
|
| 14 |
+
05/12/2026 04:16:13 - INFO - root - Num trainable parameters = 40554496
|
| 15 |
+
05/12/2026 04:20:10 - INFO - root - epochs: 1/2, steps: 50/2436, func: 0.072665, pos: 0.069513, neg: 0.14235, 2%: 3h 9m 31s
|
| 16 |
+
05/12/2026 04:24:03 - INFO - root - epochs: 1/2, steps: 100/2436, func: 0.063379, pos: 0.069799, neg: 0.087724, 4%: 3h 3m 25s
|
| 17 |
+
05/12/2026 04:27:49 - INFO - root - epochs: 1/2, steps: 150/2436, func: 0.059258, pos: 0.068181, neg: 0.093119, 6%: 2h 57m 4s
|
| 18 |
+
05/12/2026 04:31:48 - INFO - root - epochs: 1/2, steps: 200/2436, func: 0.059478, pos: 0.054782, neg: 0.055929, 8%: 2h 54m 18s
|
| 19 |
+
05/12/2026 04:35:42 - INFO - root - epochs: 1/2, steps: 250/2436, func: 0.058262, pos: 0.07347, neg: 0.062505, 10%: 2h 50m 28s
|
| 20 |
+
05/12/2026 04:39:35 - INFO - root - epochs: 1/2, steps: 300/2436, func: 0.058259, pos: 0.063938, neg: 0.046127, 12%: 2h 46m 33s
|
| 21 |
+
05/12/2026 04:43:25 - INFO - root - epochs: 1/2, steps: 350/2436, func: 0.058374, pos: 0.075189, neg: 0.036855, 14%: 2h 42m 7s
|
| 22 |
+
05/12/2026 04:47:16 - INFO - root - epochs: 1/2, steps: 400/2436, func: 0.05695, pos: 0.071096, neg: 0.030482, 16%: 2h 38m 7s
|
| 23 |
+
05/12/2026 04:51:07 - INFO - root - epochs: 1/2, steps: 450/2436, func: 0.056754, pos: 0.065966, neg: 0.034786, 18%: 2h 34m 8s
|
| 24 |
+
05/12/2026 04:54:57 - INFO - root - epochs: 1/2, steps: 500/2436, func: 0.05725, pos: 0.068398, neg: 0.0407, 20%: 2h 30m 3s
|
| 25 |
+
05/12/2026 04:58:49 - INFO - root - epochs: 1/2, steps: 550/2436, func: 0.055924, pos: 0.055691, neg: 0.03264, 22%: 2h 26m 12s
|
| 26 |
+
05/12/2026 05:02:46 - INFO - root - epochs: 1/2, steps: 600/2436, func: 0.054761, pos: 0.061301, neg: 0.027193, 24%: 2h 22m 32s
|
| 27 |
+
05/12/2026 05:06:40 - INFO - root - epochs: 1/2, steps: 650/2436, func: 0.054009, pos: 0.060621, neg: 0.03027, 26%: 2h 18m 44s
|
| 28 |
+
05/12/2026 05:10:32 - INFO - root - epochs: 1/2, steps: 700/2436, func: 0.055695, pos: 0.071853, neg: 0.028922, 28%: 2h 14m 46s
|
| 29 |
+
05/12/2026 05:14:28 - INFO - root - epochs: 1/2, steps: 750/2436, func: 0.055488, pos: 0.048709, neg: 0.04005, 30%: 2h 11m 5s
|
| 30 |
+
05/12/2026 05:18:22 - INFO - root - epochs: 1/2, steps: 800/2436, func: 0.053491, pos: 0.05827, neg: 0.025739, 32%: 2h 7m 12s
|
| 31 |
+
05/12/2026 05:22:18 - INFO - root - epochs: 1/2, steps: 850/2436, func: 0.054498, pos: 0.046493, neg: 0.019355, 34%: 2h 3m 24s
|
| 32 |
+
05/12/2026 05:26:06 - INFO - root - epochs: 1/2, steps: 900/2436, func: 0.053347, pos: 0.063695, neg: 0.033376, 36%: 1h 59m 23s
|
| 33 |
+
05/12/2026 05:29:55 - INFO - root - epochs: 1/2, steps: 950/2436, func: 0.053935, pos: 0.057924, neg: 0.022687, 38%: 1h 55m 21s
|
| 34 |
+
05/12/2026 05:33:45 - INFO - root - epochs: 1/2, steps: 1000/2436, func: 0.053086, pos: 0.042095, neg: 0.024335, 41%: 1h 51m 25s
|
| 35 |
+
05/12/2026 05:37:39 - INFO - root - epochs: 1/2, steps: 1050/2436, func: 0.054088, pos: 0.057304, neg: 0.02316, 43%: 1h 47m 36s
|
| 36 |
+
05/12/2026 05:41:29 - INFO - root - epochs: 1/2, steps: 1100/2436, func: 0.053031, pos: 0.049007, neg: 0.021122, 45%: 1h 43m 38s
|
| 37 |
+
05/12/2026 05:45:16 - INFO - root - epochs: 1/2, steps: 1150/2436, func: 0.054585, pos: 0.043532, neg: 0.02019, 47%: 1h 39m 40s
|
| 38 |
+
05/12/2026 05:49:08 - INFO - root - epochs: 1/2, steps: 1200/2436, func: 0.052821, pos: 0.050069, neg: 0.029609, 49%: 1h 35m 46s
|
| 39 |
+
05/12/2026 05:53:10 - INFO - root - epochs: 2/2, steps: 1250/2436, func: 0.053268, pos: 0.055044, neg: 0.025138, 51%: 1h 32m 3s
|
| 40 |
+
05/12/2026 05:57:00 - INFO - root - epochs: 2/2, steps: 1300/2436, func: 0.051329, pos: 0.033958, neg: 0.018176, 53%: 1h 28m 9s
|
| 41 |
+
05/12/2026 06:00:58 - INFO - root - epochs: 2/2, steps: 1350/2436, func: 0.050482, pos: 0.037176, neg: 0.022315, 55%: 1h 24m 20s
|
| 42 |
+
05/12/2026 06:04:54 - INFO - root - epochs: 2/2, steps: 1400/2436, func: 0.051296, pos: 0.04507, neg: 0.016277, 57%: 1h 20m 31s
|
| 43 |
+
05/12/2026 06:08:41 - INFO - root - epochs: 2/2, steps: 1450/2436, func: 0.052326, pos: 0.052723, neg: 0.018715, 59%: 1h 16m 32s
|
| 44 |
+
05/12/2026 06:12:36 - INFO - root - epochs: 2/2, steps: 1500/2436, func: 0.051664, pos: 0.039485, neg: 0.018645, 61%: 1h 12m 42s
|
| 45 |
+
05/12/2026 06:16:28 - INFO - root - epochs: 2/2, steps: 1550/2436, func: 0.052701, pos: 0.04412, neg: 0.014101, 63%: 1h 8m 49s
|
| 46 |
+
05/12/2026 06:20:22 - INFO - root - epochs: 2/2, steps: 1600/2436, func: 0.051173, pos: 0.039822, neg: 0.019074, 65%: 1h 4m 56s
|
| 47 |
+
05/12/2026 06:24:09 - INFO - root - epochs: 2/2, steps: 1650/2436, func: 0.052205, pos: 0.028673, neg: 0.021632, 67%: 1h 1m 1s
|
| 48 |
+
05/12/2026 06:28:07 - INFO - root - epochs: 2/2, steps: 1700/2436, func: 0.052188, pos: 0.041191, neg: 0.014831, 69%: 0h 57m 11s
|
| 49 |
+
05/12/2026 06:31:47 - INFO - root - epochs: 2/2, steps: 1750/2436, func: 0.04962, pos: 0.043797, neg: 0.01994, 71%: 0h 53m 13s
|
| 50 |
+
05/12/2026 06:35:41 - INFO - root - epochs: 2/2, steps: 1800/2436, func: 0.052593, pos: 0.048057, neg: 0.016821, 73%: 0h 49m 21s
|
| 51 |
+
05/12/2026 06:39:30 - INFO - root - epochs: 2/2, steps: 1850/2436, func: 0.050669, pos: 0.031502, neg: 0.014863, 75%: 0h 45m 27s
|
| 52 |
+
05/12/2026 06:43:21 - INFO - root - epochs: 2/2, steps: 1900/2436, func: 0.051123, pos: 0.037955, neg: 0.016594, 77%: 0h 41m 35s
|
| 53 |
+
05/12/2026 06:47:16 - INFO - root - epochs: 2/2, steps: 1950/2436, func: 0.052405, pos: 0.040127, neg: 0.016293, 80%: 0h 37m 43s
|
| 54 |
+
05/12/2026 06:51:10 - INFO - root - epochs: 2/2, steps: 2000/2436, func: 0.04957, pos: 0.03674, neg: 0.019727, 82%: 0h 33m 51s
|
| 55 |
+
05/12/2026 06:55:02 - INFO - root - epochs: 2/2, steps: 2050/2436, func: 0.050035, pos: 0.040657, neg: 0.016624, 84%: 0h 29m 59s
|
| 56 |
+
05/12/2026 06:59:01 - INFO - root - epochs: 2/2, steps: 2100/2436, func: 0.050733, pos: 0.036528, neg: 0.013117, 86%: 0h 26m 7s
|
| 57 |
+
05/12/2026 07:02:55 - INFO - root - epochs: 2/2, steps: 2150/2436, func: 0.051018, pos: 0.033595, neg: 0.021532, 88%: 0h 22m 15s
|
| 58 |
+
05/12/2026 07:06:48 - INFO - root - epochs: 2/2, steps: 2200/2436, func: 0.048103, pos: 0.042025, neg: 0.016298, 90%: 0h 18m 22s
|
| 59 |
+
05/12/2026 07:10:38 - INFO - root - epochs: 2/2, steps: 2250/2436, func: 0.050977, pos: 0.037865, neg: 0.013838, 92%: 0h 14m 29s
|
| 60 |
+
05/12/2026 07:14:24 - INFO - root - epochs: 2/2, steps: 2300/2436, func: 0.050455, pos: 0.036463, neg: 0.012117, 94%: 0h 10m 36s
|
| 61 |
+
05/12/2026 07:18:16 - INFO - root - epochs: 2/2, steps: 2350/2436, func: 0.05013, pos: 0.034424, neg: 0.018696, 96%: 0h 6m 44s
|
| 62 |
+
05/12/2026 07:22:11 - INFO - root - epochs: 2/2, steps: 2400/2436, func: 0.05218, pos: 0.038016, neg: 0.015734, 98%: 0h 2m 52s
|
| 63 |
+
05/12/2026 07:29:57 - INFO - root - final eval loss: func: 0.052527, pos: 0.063118, neg: 0.035372
|
| 64 |
+
05/12/2026 07:29:57 - INFO - root - Saving model checkpoint to ../trained/llama2-7b-lora-safecoder/checkpoint-last
|
mistral-7b-lora-safecoder/checkpoint-last/README.md
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: mistralai/Mistral-7B-v0.1
|
| 3 |
+
library_name: peft
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Model Card for Model ID
|
| 7 |
+
|
| 8 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
## Model Details
|
| 13 |
+
|
| 14 |
+
### Model Description
|
| 15 |
+
|
| 16 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
- **Developed by:** [More Information Needed]
|
| 21 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 22 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 23 |
+
- **Model type:** [More Information Needed]
|
| 24 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 25 |
+
- **License:** [More Information Needed]
|
| 26 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 27 |
+
|
| 28 |
+
### Model Sources [optional]
|
| 29 |
+
|
| 30 |
+
<!-- Provide the basic links for the model. -->
|
| 31 |
+
|
| 32 |
+
- **Repository:** [More Information Needed]
|
| 33 |
+
- **Paper [optional]:** [More Information Needed]
|
| 34 |
+
- **Demo [optional]:** [More Information Needed]
|
| 35 |
+
|
| 36 |
+
## Uses
|
| 37 |
+
|
| 38 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 39 |
+
|
| 40 |
+
### Direct Use
|
| 41 |
+
|
| 42 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 43 |
+
|
| 44 |
+
[More Information Needed]
|
| 45 |
+
|
| 46 |
+
### Downstream Use [optional]
|
| 47 |
+
|
| 48 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 49 |
+
|
| 50 |
+
[More Information Needed]
|
| 51 |
+
|
| 52 |
+
### Out-of-Scope Use
|
| 53 |
+
|
| 54 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 55 |
+
|
| 56 |
+
[More Information Needed]
|
| 57 |
+
|
| 58 |
+
## Bias, Risks, and Limitations
|
| 59 |
+
|
| 60 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 61 |
+
|
| 62 |
+
[More Information Needed]
|
| 63 |
+
|
| 64 |
+
### Recommendations
|
| 65 |
+
|
| 66 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 67 |
+
|
| 68 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 69 |
+
|
| 70 |
+
## How to Get Started with the Model
|
| 71 |
+
|
| 72 |
+
Use the code below to get started with the model.
|
| 73 |
+
|
| 74 |
+
[More Information Needed]
|
| 75 |
+
|
| 76 |
+
## Training Details
|
| 77 |
+
|
| 78 |
+
### Training Data
|
| 79 |
+
|
| 80 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 81 |
+
|
| 82 |
+
[More Information Needed]
|
| 83 |
+
|
| 84 |
+
### Training Procedure
|
| 85 |
+
|
| 86 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 87 |
+
|
| 88 |
+
#### Preprocessing [optional]
|
| 89 |
+
|
| 90 |
+
[More Information Needed]
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
#### Training Hyperparameters
|
| 94 |
+
|
| 95 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 96 |
+
|
| 97 |
+
#### Speeds, Sizes, Times [optional]
|
| 98 |
+
|
| 99 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 100 |
+
|
| 101 |
+
[More Information Needed]
|
| 102 |
+
|
| 103 |
+
## Evaluation
|
| 104 |
+
|
| 105 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 106 |
+
|
| 107 |
+
### Testing Data, Factors & Metrics
|
| 108 |
+
|
| 109 |
+
#### Testing Data
|
| 110 |
+
|
| 111 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 112 |
+
|
| 113 |
+
[More Information Needed]
|
| 114 |
+
|
| 115 |
+
#### Factors
|
| 116 |
+
|
| 117 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 118 |
+
|
| 119 |
+
[More Information Needed]
|
| 120 |
+
|
| 121 |
+
#### Metrics
|
| 122 |
+
|
| 123 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 124 |
+
|
| 125 |
+
[More Information Needed]
|
| 126 |
+
|
| 127 |
+
### Results
|
| 128 |
+
|
| 129 |
+
[More Information Needed]
|
| 130 |
+
|
| 131 |
+
#### Summary
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
## Model Examination [optional]
|
| 136 |
+
|
| 137 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 138 |
+
|
| 139 |
+
[More Information Needed]
|
| 140 |
+
|
| 141 |
+
## Environmental Impact
|
| 142 |
+
|
| 143 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 144 |
+
|
| 145 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 146 |
+
|
| 147 |
+
- **Hardware Type:** [More Information Needed]
|
| 148 |
+
- **Hours used:** [More Information Needed]
|
| 149 |
+
- **Cloud Provider:** [More Information Needed]
|
| 150 |
+
- **Compute Region:** [More Information Needed]
|
| 151 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 152 |
+
|
| 153 |
+
## Technical Specifications [optional]
|
| 154 |
+
|
| 155 |
+
### Model Architecture and Objective
|
| 156 |
+
|
| 157 |
+
[More Information Needed]
|
| 158 |
+
|
| 159 |
+
### Compute Infrastructure
|
| 160 |
+
|
| 161 |
+
[More Information Needed]
|
| 162 |
+
|
| 163 |
+
#### Hardware
|
| 164 |
+
|
| 165 |
+
[More Information Needed]
|
| 166 |
+
|
| 167 |
+
#### Software
|
| 168 |
+
|
| 169 |
+
[More Information Needed]
|
| 170 |
+
|
| 171 |
+
## Citation [optional]
|
| 172 |
+
|
| 173 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 174 |
+
|
| 175 |
+
**BibTeX:**
|
| 176 |
+
|
| 177 |
+
[More Information Needed]
|
| 178 |
+
|
| 179 |
+
**APA:**
|
| 180 |
+
|
| 181 |
+
[More Information Needed]
|
| 182 |
+
|
| 183 |
+
## Glossary [optional]
|
| 184 |
+
|
| 185 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 186 |
+
|
| 187 |
+
[More Information Needed]
|
| 188 |
+
|
| 189 |
+
## More Information [optional]
|
| 190 |
+
|
| 191 |
+
[More Information Needed]
|
| 192 |
+
|
| 193 |
+
## Model Card Authors [optional]
|
| 194 |
+
|
| 195 |
+
[More Information Needed]
|
| 196 |
+
|
| 197 |
+
## Model Card Contact
|
| 198 |
+
|
| 199 |
+
[More Information Needed]
|
| 200 |
+
### Framework versions
|
| 201 |
+
|
| 202 |
+
- PEFT 0.8.2
|
mistral-7b-lora-safecoder/checkpoint-last/adapter_config.json
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alpha_pattern": {},
|
| 3 |
+
"auto_mapping": null,
|
| 4 |
+
"base_model_name_or_path": "mistralai/Mistral-7B-v0.1",
|
| 5 |
+
"bias": "none",
|
| 6 |
+
"fan_in_fan_out": false,
|
| 7 |
+
"inference_mode": true,
|
| 8 |
+
"init_lora_weights": true,
|
| 9 |
+
"layers_pattern": null,
|
| 10 |
+
"layers_to_transform": null,
|
| 11 |
+
"loftq_config": {},
|
| 12 |
+
"lora_alpha": 32,
|
| 13 |
+
"lora_dropout": 0.1,
|
| 14 |
+
"megatron_config": null,
|
| 15 |
+
"megatron_core": "megatron.core",
|
| 16 |
+
"modules_to_save": null,
|
| 17 |
+
"peft_type": "LORA",
|
| 18 |
+
"r": 16,
|
| 19 |
+
"rank_pattern": {},
|
| 20 |
+
"revision": null,
|
| 21 |
+
"target_modules": [
|
| 22 |
+
"lm_head",
|
| 23 |
+
"v_proj",
|
| 24 |
+
"q_proj",
|
| 25 |
+
"gate_proj",
|
| 26 |
+
"k_proj",
|
| 27 |
+
"o_proj",
|
| 28 |
+
"down_proj",
|
| 29 |
+
"up_proj"
|
| 30 |
+
],
|
| 31 |
+
"task_type": "CAUSAL_LM",
|
| 32 |
+
"use_rslora": false
|
| 33 |
+
}
|
mistral-7b-lora-safecoder/checkpoint-last/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f65ad0b0a2655ee6299b7e820f19dd70f8e2e0e6bf00ebaca8bc4ac9bb7e33ea
|
| 3 |
+
size 694431312
|
mistral-7b-lora-safecoder/checkpoint-last/special_tokens_map.json
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token": {
|
| 3 |
+
"content": "<s>",
|
| 4 |
+
"lstrip": false,
|
| 5 |
+
"normalized": false,
|
| 6 |
+
"rstrip": false,
|
| 7 |
+
"single_word": false
|
| 8 |
+
},
|
| 9 |
+
"eos_token": {
|
| 10 |
+
"content": "</s>",
|
| 11 |
+
"lstrip": false,
|
| 12 |
+
"normalized": false,
|
| 13 |
+
"rstrip": false,
|
| 14 |
+
"single_word": false
|
| 15 |
+
},
|
| 16 |
+
"unk_token": {
|
| 17 |
+
"content": "<unk>",
|
| 18 |
+
"lstrip": false,
|
| 19 |
+
"normalized": false,
|
| 20 |
+
"rstrip": false,
|
| 21 |
+
"single_word": false
|
| 22 |
+
}
|
| 23 |
+
}
|
mistral-7b-lora-safecoder/checkpoint-last/tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
mistral-7b-lora-safecoder/checkpoint-last/tokenizer.model
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
|
| 3 |
+
size 493443
|
mistral-7b-lora-safecoder/checkpoint-last/tokenizer_config.json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_bos_token": true,
|
| 3 |
+
"add_eos_token": false,
|
| 4 |
+
"add_prefix_space": null,
|
| 5 |
+
"added_tokens_decoder": {
|
| 6 |
+
"0": {
|
| 7 |
+
"content": "<unk>",
|
| 8 |
+
"lstrip": false,
|
| 9 |
+
"normalized": false,
|
| 10 |
+
"rstrip": false,
|
| 11 |
+
"single_word": false,
|
| 12 |
+
"special": true
|
| 13 |
+
},
|
| 14 |
+
"1": {
|
| 15 |
+
"content": "<s>",
|
| 16 |
+
"lstrip": false,
|
| 17 |
+
"normalized": false,
|
| 18 |
+
"rstrip": false,
|
| 19 |
+
"single_word": false,
|
| 20 |
+
"special": true
|
| 21 |
+
},
|
| 22 |
+
"2": {
|
| 23 |
+
"content": "</s>",
|
| 24 |
+
"lstrip": false,
|
| 25 |
+
"normalized": false,
|
| 26 |
+
"rstrip": false,
|
| 27 |
+
"single_word": false,
|
| 28 |
+
"special": true
|
| 29 |
+
}
|
| 30 |
+
},
|
| 31 |
+
"additional_special_tokens": [],
|
| 32 |
+
"bos_token": "<s>",
|
| 33 |
+
"clean_up_tokenization_spaces": false,
|
| 34 |
+
"eos_token": "</s>",
|
| 35 |
+
"legacy": false,
|
| 36 |
+
"model_max_length": 1000000000000000019884624838656,
|
| 37 |
+
"pad_token": null,
|
| 38 |
+
"sp_model_kwargs": {},
|
| 39 |
+
"spaces_between_special_tokens": false,
|
| 40 |
+
"tokenizer_class": "LlamaTokenizer",
|
| 41 |
+
"unk_token": "<unk>",
|
| 42 |
+
"use_default_system_prompt": false
|
| 43 |
+
}
|
mistral-7b-lora-safecoder/train.log
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
05/06/2026 17:42:43 - INFO - accelerate.utils.modeling - We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can set `max_memory` in to a higher value to use more memory (at your own risk).
|
| 2 |
+
05/06/2026 17:43:36 - INFO - root - number of sec samples before upsampling: 1809
|
| 3 |
+
05/06/2026 17:43:36 - INFO - root - number of sec samples after upsampling: 3347
|
| 4 |
+
05/06/2026 17:43:38 - INFO - root - Training args Namespace(output_name='mistral-7b-lora-safecoder', datasets=['lmsys', 'sec-desc', 'sec-new-desc'], pretrain_name='mistral-7b', loss_weight=1.0, sven=False, num_train_epochs=2, learning_rate=2e-05, max_num_tokens=1024, batch_size=1, grad_acc_steps=16, weight_decay=0.01, adam_epsilon=1e-08, warmup_steps=0, max_grad_norm=1.0, dropout=0.1, kl_loss_weight=0, exclude_neg=False, no_weights=False, lora=True, r=16, lora_alpha=32, lora_dropout=0.1, sampling_size=40, sampling_method='minority', cwes=['all'], langs=['all'], logging_steps=50, save_epochs=10, seed=2, data_dir='../data_train_val', model_dir='../trained/', output_dir='../trained/mistral-7b-lora-safecoder', logger=<RootLogger root (INFO)>)
|
| 5 |
+
05/06/2026 17:43:38 - INFO - root - ***** Running training *****
|
| 6 |
+
05/06/2026 17:43:38 - INFO - root - Num samples = 19568
|
| 7 |
+
05/06/2026 17:43:38 - INFO - root - Num epoch = 2
|
| 8 |
+
05/06/2026 17:43:38 - INFO - root - Batch size= 1
|
| 9 |
+
05/06/2026 17:43:38 - INFO - root - Total batch size (w. accumulation) = 16
|
| 10 |
+
05/06/2026 17:43:38 - INFO - root - Gradient Accumulation steps = 16
|
| 11 |
+
05/06/2026 17:43:38 - INFO - root - Total optimization steps = 2446
|
| 12 |
+
05/06/2026 17:43:38 - INFO - root - Num val samples = 1999
|
| 13 |
+
05/06/2026 17:43:38 - INFO - root - Num parameters = 7284252672
|
| 14 |
+
05/06/2026 17:43:38 - INFO - root - Num trainable parameters = 42520576
|
| 15 |
+
05/06/2026 17:47:25 - INFO - root - epochs: 1/2, steps: 50/2446, func: 0.060247, pos: 0.072874, neg: 0.11132, 2%: 3h 2m 7s
|
| 16 |
+
05/06/2026 17:51:12 - INFO - root - epochs: 1/2, steps: 100/2446, func: 0.055213, pos: 0.075717, neg: 0.063684, 4%: 2h 57m 35s
|
| 17 |
+
05/06/2026 17:54:58 - INFO - root - epochs: 1/2, steps: 150/2446, func: 0.05299, pos: 0.073642, neg: 0.055921, 6%: 2h 53m 32s
|
| 18 |
+
05/06/2026 17:58:50 - INFO - root - epochs: 1/2, steps: 200/2446, func: 0.051302, pos: 0.056611, neg: 0.056372, 8%: 2h 51m 1s
|
| 19 |
+
05/06/2026 18:02:35 - INFO - root - epochs: 1/2, steps: 250/2446, func: 0.051405, pos: 0.05595, neg: 0.040865, 10%: 2h 46m 28s
|
| 20 |
+
05/06/2026 18:06:21 - INFO - root - epochs: 1/2, steps: 300/2446, func: 0.050281, pos: 0.053411, neg: 0.045239, 12%: 2h 42m 33s
|
| 21 |
+
05/06/2026 18:10:07 - INFO - root - epochs: 1/2, steps: 350/2446, func: 0.051736, pos: 0.050501, neg: 0.029558, 14%: 2h 38m 42s
|
| 22 |
+
05/06/2026 18:13:49 - INFO - root - epochs: 1/2, steps: 400/2446, func: 0.049518, pos: 0.044457, neg: 0.042594, 16%: 2h 34m 26s
|
| 23 |
+
05/06/2026 18:17:31 - INFO - root - epochs: 1/2, steps: 450/2446, func: 0.051457, pos: 0.055581, neg: 0.037253, 18%: 2h 30m 16s
|
| 24 |
+
05/06/2026 18:21:15 - INFO - root - epochs: 1/2, steps: 500/2446, func: 0.049959, pos: 0.04995, neg: 0.030102, 20%: 2h 26m 29s
|
| 25 |
+
05/06/2026 18:24:56 - INFO - root - epochs: 1/2, steps: 550/2446, func: 0.048783, pos: 0.051764, neg: 0.027316, 22%: 2h 22m 25s
|
| 26 |
+
05/06/2026 18:28:40 - INFO - root - epochs: 1/2, steps: 600/2446, func: 0.049506, pos: 0.034509, neg: 0.031578, 24%: 2h 18m 39s
|
| 27 |
+
05/06/2026 18:32:28 - INFO - root - epochs: 1/2, steps: 650/2446, func: 0.049017, pos: 0.057761, neg: 0.021366, 26%: 2h 15m 0s
|
| 28 |
+
05/06/2026 18:36:04 - INFO - root - epochs: 1/2, steps: 700/2446, func: 0.049175, pos: 0.051523, neg: 0.032127, 28%: 2h 10m 52s
|
| 29 |
+
05/06/2026 18:39:59 - INFO - root - epochs: 1/2, steps: 750/2446, func: 0.048054, pos: 0.045201, neg: 0.024674, 30%: 2h 7m 33s
|
| 30 |
+
05/06/2026 18:43:38 - INFO - root - epochs: 1/2, steps: 800/2446, func: 0.047632, pos: 0.062736, neg: 0.018429, 32%: 2h 3m 31s
|
| 31 |
+
05/06/2026 18:47:28 - INFO - root - epochs: 1/2, steps: 850/2446, func: 0.047464, pos: 0.034862, neg: 0.026122, 34%: 1h 59m 58s
|
| 32 |
+
05/06/2026 18:51:22 - INFO - root - epochs: 1/2, steps: 900/2446, func: 0.047728, pos: 0.047329, neg: 0.024224, 36%: 1h 56m 27s
|
| 33 |
+
05/06/2026 18:55:11 - INFO - root - epochs: 1/2, steps: 950/2446, func: 0.045727, pos: 0.039367, neg: 0.016257, 38%: 1h 52m 45s
|
| 34 |
+
05/06/2026 18:58:58 - INFO - root - epochs: 1/2, steps: 1000/2446, func: 0.046647, pos: 0.037901, neg: 0.022523, 40%: 1h 49m 1s
|
| 35 |
+
05/06/2026 19:02:45 - INFO - root - epochs: 1/2, steps: 1050/2446, func: 0.047766, pos: 0.042613, neg: 0.026214, 42%: 1h 45m 16s
|
| 36 |
+
05/06/2026 19:06:31 - INFO - root - epochs: 1/2, steps: 1100/2446, func: 0.048064, pos: 0.034459, neg: 0.021552, 44%: 1h 41m 29s
|
| 37 |
+
05/06/2026 19:10:20 - INFO - root - epochs: 1/2, steps: 1150/2446, func: 0.048113, pos: 0.0386, neg: 0.015921, 46%: 1h 37m 47s
|
| 38 |
+
05/06/2026 19:14:05 - INFO - root - epochs: 1/2, steps: 1200/2446, func: 0.048246, pos: 0.040583, neg: 0.01884, 49%: 1h 34m 0s
|
| 39 |
+
05/06/2026 19:17:54 - INFO - root - epochs: 2/2, steps: 1250/2446, func: 0.045884, pos: 0.030382, neg: 0.012033, 51%: 1h 30m 16s
|
| 40 |
+
05/06/2026 19:21:39 - INFO - root - epochs: 2/2, steps: 1300/2446, func: 0.043194, pos: 0.044291, neg: 0.01279, 53%: 1h 26m 29s
|
| 41 |
+
05/06/2026 19:25:25 - INFO - root - epochs: 2/2, steps: 1350/2446, func: 0.044463, pos: 0.019906, neg: 0.011339, 55%: 1h 22m 42s
|
| 42 |
+
05/06/2026 19:29:14 - INFO - root - epochs: 2/2, steps: 1400/2446, func: 0.044169, pos: 0.032827, neg: 0.009273, 57%: 1h 18m 58s
|
| 43 |
+
05/06/2026 19:33:06 - INFO - root - epochs: 2/2, steps: 1450/2446, func: 0.044901, pos: 0.029661, neg: 0.011736, 59%: 1h 15m 15s
|
| 44 |
+
05/06/2026 19:36:59 - INFO - root - epochs: 2/2, steps: 1500/2446, func: 0.044587, pos: 0.026535, neg: 0.010645, 61%: 1h 11m 33s
|
| 45 |
+
05/06/2026 19:40:44 - INFO - root - epochs: 2/2, steps: 1550/2446, func: 0.043455, pos: 0.029962, neg: 0.008954, 63%: 1h 7m 46s
|
| 46 |
+
05/06/2026 19:44:33 - INFO - root - epochs: 2/2, steps: 1600/2446, func: 0.045786, pos: 0.031509, neg: 0.010147, 65%: 1h 4m 1s
|
| 47 |
+
05/06/2026 19:48:22 - INFO - root - epochs: 2/2, steps: 1650/2446, func: 0.044594, pos: 0.022883, neg: 0.008734, 67%: 1h 0m 14s
|
| 48 |
+
05/06/2026 19:52:06 - INFO - root - epochs: 2/2, steps: 1700/2446, func: 0.043063, pos: 0.030558, neg: 0.012907, 69%: 0h 56m 27s
|
| 49 |
+
05/06/2026 19:55:54 - INFO - root - epochs: 2/2, steps: 1750/2446, func: 0.044163, pos: 0.024725, neg: 0.011875, 71%: 0h 52m 41s
|
| 50 |
+
05/06/2026 19:59:46 - INFO - root - epochs: 2/2, steps: 1800/2446, func: 0.042532, pos: 0.027732, neg: 0.008819, 73%: 0h 48m 55s
|
| 51 |
+
05/06/2026 20:03:36 - INFO - root - epochs: 2/2, steps: 1850/2446, func: 0.044665, pos: 0.038731, neg: 0.013127, 75%: 0h 45m 9s
|
| 52 |
+
05/06/2026 20:07:25 - INFO - root - epochs: 2/2, steps: 1900/2446, func: 0.042714, pos: 0.023254, neg: 0.009168, 77%: 0h 41m 23s
|
| 53 |
+
05/06/2026 20:11:08 - INFO - root - epochs: 2/2, steps: 1950/2446, func: 0.041709, pos: 0.027533, neg: 0.008855, 79%: 0h 37m 35s
|
| 54 |
+
05/06/2026 20:14:55 - INFO - root - epochs: 2/2, steps: 2000/2446, func: 0.043733, pos: 0.023785, neg: 0.007754, 81%: 0h 33m 48s
|
| 55 |
+
05/06/2026 20:18:34 - INFO - root - epochs: 2/2, steps: 2050/2446, func: 0.041731, pos: 0.027958, neg: 0.012908, 83%: 0h 30m 0s
|
| 56 |
+
05/06/2026 20:22:23 - INFO - root - epochs: 2/2, steps: 2100/2446, func: 0.043956, pos: 0.021331, neg: 0.011778, 85%: 0h 26m 13s
|
| 57 |
+
05/06/2026 20:26:10 - INFO - root - epochs: 2/2, steps: 2150/2446, func: 0.043338, pos: 0.026119, neg: 0.010473, 87%: 0h 22m 27s
|
| 58 |
+
05/06/2026 20:29:57 - INFO - root - epochs: 2/2, steps: 2200/2446, func: 0.043766, pos: 0.021158, neg: 0.010124, 89%: 0h 18m 40s
|
| 59 |
+
05/06/2026 20:33:41 - INFO - root - epochs: 2/2, steps: 2250/2446, func: 0.042976, pos: 0.026389, neg: 0.018828, 91%: 0h 14m 53s
|
| 60 |
+
05/06/2026 20:37:27 - INFO - root - epochs: 2/2, steps: 2300/2446, func: 0.044326, pos: 0.030221, neg: 0.011533, 93%: 0h 11m 6s
|
| 61 |
+
05/06/2026 20:41:24 - INFO - root - epochs: 2/2, steps: 2350/2446, func: 0.04236, pos: 0.031234, neg: 0.01894, 96%: 0h 7m 20s
|
| 62 |
+
05/06/2026 20:45:09 - INFO - root - epochs: 2/2, steps: 2400/2446, func: 0.044176, pos: 0.018846, neg: 0.011987, 98%: 0h 3m 33s
|
| 63 |
+
05/06/2026 20:53:04 - INFO - root - final eval loss: func: 0.046159, pos: 0.053182, neg: 0.04193
|
| 64 |
+
05/06/2026 20:53:04 - INFO - root - Saving model checkpoint to ../trained/mistral-7b-lora-safecoder/checkpoint-last
|
phi-2-lora-safecoder/checkpoint-last/README.md
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: microsoft/phi-2
|
| 3 |
+
library_name: peft
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Model Card for Model ID
|
| 7 |
+
|
| 8 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
## Model Details
|
| 13 |
+
|
| 14 |
+
### Model Description
|
| 15 |
+
|
| 16 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
- **Developed by:** [More Information Needed]
|
| 21 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 22 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 23 |
+
- **Model type:** [More Information Needed]
|
| 24 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 25 |
+
- **License:** [More Information Needed]
|
| 26 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 27 |
+
|
| 28 |
+
### Model Sources [optional]
|
| 29 |
+
|
| 30 |
+
<!-- Provide the basic links for the model. -->
|
| 31 |
+
|
| 32 |
+
- **Repository:** [More Information Needed]
|
| 33 |
+
- **Paper [optional]:** [More Information Needed]
|
| 34 |
+
- **Demo [optional]:** [More Information Needed]
|
| 35 |
+
|
| 36 |
+
## Uses
|
| 37 |
+
|
| 38 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 39 |
+
|
| 40 |
+
### Direct Use
|
| 41 |
+
|
| 42 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 43 |
+
|
| 44 |
+
[More Information Needed]
|
| 45 |
+
|
| 46 |
+
### Downstream Use [optional]
|
| 47 |
+
|
| 48 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 49 |
+
|
| 50 |
+
[More Information Needed]
|
| 51 |
+
|
| 52 |
+
### Out-of-Scope Use
|
| 53 |
+
|
| 54 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 55 |
+
|
| 56 |
+
[More Information Needed]
|
| 57 |
+
|
| 58 |
+
## Bias, Risks, and Limitations
|
| 59 |
+
|
| 60 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 61 |
+
|
| 62 |
+
[More Information Needed]
|
| 63 |
+
|
| 64 |
+
### Recommendations
|
| 65 |
+
|
| 66 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 67 |
+
|
| 68 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 69 |
+
|
| 70 |
+
## How to Get Started with the Model
|
| 71 |
+
|
| 72 |
+
Use the code below to get started with the model.
|
| 73 |
+
|
| 74 |
+
[More Information Needed]
|
| 75 |
+
|
| 76 |
+
## Training Details
|
| 77 |
+
|
| 78 |
+
### Training Data
|
| 79 |
+
|
| 80 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 81 |
+
|
| 82 |
+
[More Information Needed]
|
| 83 |
+
|
| 84 |
+
### Training Procedure
|
| 85 |
+
|
| 86 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 87 |
+
|
| 88 |
+
#### Preprocessing [optional]
|
| 89 |
+
|
| 90 |
+
[More Information Needed]
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
#### Training Hyperparameters
|
| 94 |
+
|
| 95 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 96 |
+
|
| 97 |
+
#### Speeds, Sizes, Times [optional]
|
| 98 |
+
|
| 99 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 100 |
+
|
| 101 |
+
[More Information Needed]
|
| 102 |
+
|
| 103 |
+
## Evaluation
|
| 104 |
+
|
| 105 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 106 |
+
|
| 107 |
+
### Testing Data, Factors & Metrics
|
| 108 |
+
|
| 109 |
+
#### Testing Data
|
| 110 |
+
|
| 111 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 112 |
+
|
| 113 |
+
[More Information Needed]
|
| 114 |
+
|
| 115 |
+
#### Factors
|
| 116 |
+
|
| 117 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 118 |
+
|
| 119 |
+
[More Information Needed]
|
| 120 |
+
|
| 121 |
+
#### Metrics
|
| 122 |
+
|
| 123 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 124 |
+
|
| 125 |
+
[More Information Needed]
|
| 126 |
+
|
| 127 |
+
### Results
|
| 128 |
+
|
| 129 |
+
[More Information Needed]
|
| 130 |
+
|
| 131 |
+
#### Summary
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
## Model Examination [optional]
|
| 136 |
+
|
| 137 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 138 |
+
|
| 139 |
+
[More Information Needed]
|
| 140 |
+
|
| 141 |
+
## Environmental Impact
|
| 142 |
+
|
| 143 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 144 |
+
|
| 145 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 146 |
+
|
| 147 |
+
- **Hardware Type:** [More Information Needed]
|
| 148 |
+
- **Hours used:** [More Information Needed]
|
| 149 |
+
- **Cloud Provider:** [More Information Needed]
|
| 150 |
+
- **Compute Region:** [More Information Needed]
|
| 151 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 152 |
+
|
| 153 |
+
## Technical Specifications [optional]
|
| 154 |
+
|
| 155 |
+
### Model Architecture and Objective
|
| 156 |
+
|
| 157 |
+
[More Information Needed]
|
| 158 |
+
|
| 159 |
+
### Compute Infrastructure
|
| 160 |
+
|
| 161 |
+
[More Information Needed]
|
| 162 |
+
|
| 163 |
+
#### Hardware
|
| 164 |
+
|
| 165 |
+
[More Information Needed]
|
| 166 |
+
|
| 167 |
+
#### Software
|
| 168 |
+
|
| 169 |
+
[More Information Needed]
|
| 170 |
+
|
| 171 |
+
## Citation [optional]
|
| 172 |
+
|
| 173 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 174 |
+
|
| 175 |
+
**BibTeX:**
|
| 176 |
+
|
| 177 |
+
[More Information Needed]
|
| 178 |
+
|
| 179 |
+
**APA:**
|
| 180 |
+
|
| 181 |
+
[More Information Needed]
|
| 182 |
+
|
| 183 |
+
## Glossary [optional]
|
| 184 |
+
|
| 185 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 186 |
+
|
| 187 |
+
[More Information Needed]
|
| 188 |
+
|
| 189 |
+
## More Information [optional]
|
| 190 |
+
|
| 191 |
+
[More Information Needed]
|
| 192 |
+
|
| 193 |
+
## Model Card Authors [optional]
|
| 194 |
+
|
| 195 |
+
[More Information Needed]
|
| 196 |
+
|
| 197 |
+
## Model Card Contact
|
| 198 |
+
|
| 199 |
+
[More Information Needed]
|
| 200 |
+
### Framework versions
|
| 201 |
+
|
| 202 |
+
- PEFT 0.8.2
|
phi-2-lora-safecoder/checkpoint-last/adapter_config.json
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alpha_pattern": {},
|
| 3 |
+
"auto_mapping": null,
|
| 4 |
+
"base_model_name_or_path": "microsoft/phi-2",
|
| 5 |
+
"bias": "none",
|
| 6 |
+
"fan_in_fan_out": false,
|
| 7 |
+
"inference_mode": true,
|
| 8 |
+
"init_lora_weights": true,
|
| 9 |
+
"layers_pattern": null,
|
| 10 |
+
"layers_to_transform": null,
|
| 11 |
+
"loftq_config": {},
|
| 12 |
+
"lora_alpha": 32,
|
| 13 |
+
"lora_dropout": 0.1,
|
| 14 |
+
"megatron_config": null,
|
| 15 |
+
"megatron_core": "megatron.core",
|
| 16 |
+
"modules_to_save": null,
|
| 17 |
+
"peft_type": "LORA",
|
| 18 |
+
"r": 16,
|
| 19 |
+
"rank_pattern": {},
|
| 20 |
+
"revision": null,
|
| 21 |
+
"target_modules": [
|
| 22 |
+
"dense",
|
| 23 |
+
"q_proj",
|
| 24 |
+
"lm_head",
|
| 25 |
+
"fc1",
|
| 26 |
+
"k_proj",
|
| 27 |
+
"fc2",
|
| 28 |
+
"v_proj"
|
| 29 |
+
],
|
| 30 |
+
"task_type": "CAUSAL_LM",
|
| 31 |
+
"use_rslora": false
|
| 32 |
+
}
|
phi-2-lora-safecoder/checkpoint-last/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a5482769430dae93c41c01e995a441902262ca14a95e69cda5d12e166ed0dc41
|
| 3 |
+
size 613028364
|
phi-2-lora-safecoder/checkpoint-last/added_tokens.json
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"\t\t": 50294,
|
| 3 |
+
"\t\t\t": 50293,
|
| 4 |
+
"\t\t\t\t": 50292,
|
| 5 |
+
"\t\t\t\t\t": 50291,
|
| 6 |
+
"\t\t\t\t\t\t": 50290,
|
| 7 |
+
"\t\t\t\t\t\t\t": 50289,
|
| 8 |
+
"\t\t\t\t\t\t\t\t": 50288,
|
| 9 |
+
"\t\t\t\t\t\t\t\t\t": 50287,
|
| 10 |
+
" ": 50286,
|
| 11 |
+
" ": 50285,
|
| 12 |
+
" ": 50284,
|
| 13 |
+
" ": 50283,
|
| 14 |
+
" ": 50282,
|
| 15 |
+
" ": 50281,
|
| 16 |
+
" ": 50280,
|
| 17 |
+
" ": 50279,
|
| 18 |
+
" ": 50278,
|
| 19 |
+
" ": 50277,
|
| 20 |
+
" ": 50276,
|
| 21 |
+
" ": 50275,
|
| 22 |
+
" ": 50274,
|
| 23 |
+
" ": 50273,
|
| 24 |
+
" ": 50272,
|
| 25 |
+
" ": 50271,
|
| 26 |
+
" ": 50270,
|
| 27 |
+
" ": 50269,
|
| 28 |
+
" ": 50268,
|
| 29 |
+
" ": 50267,
|
| 30 |
+
" ": 50266,
|
| 31 |
+
" ": 50265,
|
| 32 |
+
" ": 50264,
|
| 33 |
+
" ": 50263,
|
| 34 |
+
" ": 50262,
|
| 35 |
+
" ": 50261,
|
| 36 |
+
" ": 50260,
|
| 37 |
+
" ": 50259,
|
| 38 |
+
" ": 50258,
|
| 39 |
+
" ": 50257
|
| 40 |
+
}
|
phi-2-lora-safecoder/checkpoint-last/merges.txt
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
phi-2-lora-safecoder/checkpoint-last/special_tokens_map.json
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token": {
|
| 3 |
+
"content": "<|endoftext|>",
|
| 4 |
+
"lstrip": false,
|
| 5 |
+
"normalized": false,
|
| 6 |
+
"rstrip": false,
|
| 7 |
+
"single_word": false
|
| 8 |
+
},
|
| 9 |
+
"eos_token": {
|
| 10 |
+
"content": "<|endoftext|>",
|
| 11 |
+
"lstrip": false,
|
| 12 |
+
"normalized": false,
|
| 13 |
+
"rstrip": false,
|
| 14 |
+
"single_word": false
|
| 15 |
+
},
|
| 16 |
+
"unk_token": {
|
| 17 |
+
"content": "<|endoftext|>",
|
| 18 |
+
"lstrip": false,
|
| 19 |
+
"normalized": false,
|
| 20 |
+
"rstrip": false,
|
| 21 |
+
"single_word": false
|
| 22 |
+
}
|
| 23 |
+
}
|
phi-2-lora-safecoder/checkpoint-last/tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
phi-2-lora-safecoder/checkpoint-last/tokenizer_config.json
ADDED
|
@@ -0,0 +1,325 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_prefix_space": false,
|
| 3 |
+
"added_tokens_decoder": {
|
| 4 |
+
"50256": {
|
| 5 |
+
"content": "<|endoftext|>",
|
| 6 |
+
"lstrip": false,
|
| 7 |
+
"normalized": false,
|
| 8 |
+
"rstrip": false,
|
| 9 |
+
"single_word": false,
|
| 10 |
+
"special": true
|
| 11 |
+
},
|
| 12 |
+
"50257": {
|
| 13 |
+
"content": " ",
|
| 14 |
+
"lstrip": false,
|
| 15 |
+
"normalized": true,
|
| 16 |
+
"rstrip": false,
|
| 17 |
+
"single_word": false,
|
| 18 |
+
"special": false
|
| 19 |
+
},
|
| 20 |
+
"50258": {
|
| 21 |
+
"content": " ",
|
| 22 |
+
"lstrip": false,
|
| 23 |
+
"normalized": true,
|
| 24 |
+
"rstrip": false,
|
| 25 |
+
"single_word": false,
|
| 26 |
+
"special": false
|
| 27 |
+
},
|
| 28 |
+
"50259": {
|
| 29 |
+
"content": " ",
|
| 30 |
+
"lstrip": false,
|
| 31 |
+
"normalized": true,
|
| 32 |
+
"rstrip": false,
|
| 33 |
+
"single_word": false,
|
| 34 |
+
"special": false
|
| 35 |
+
},
|
| 36 |
+
"50260": {
|
| 37 |
+
"content": " ",
|
| 38 |
+
"lstrip": false,
|
| 39 |
+
"normalized": true,
|
| 40 |
+
"rstrip": false,
|
| 41 |
+
"single_word": false,
|
| 42 |
+
"special": false
|
| 43 |
+
},
|
| 44 |
+
"50261": {
|
| 45 |
+
"content": " ",
|
| 46 |
+
"lstrip": false,
|
| 47 |
+
"normalized": true,
|
| 48 |
+
"rstrip": false,
|
| 49 |
+
"single_word": false,
|
| 50 |
+
"special": false
|
| 51 |
+
},
|
| 52 |
+
"50262": {
|
| 53 |
+
"content": " ",
|
| 54 |
+
"lstrip": false,
|
| 55 |
+
"normalized": true,
|
| 56 |
+
"rstrip": false,
|
| 57 |
+
"single_word": false,
|
| 58 |
+
"special": false
|
| 59 |
+
},
|
| 60 |
+
"50263": {
|
| 61 |
+
"content": " ",
|
| 62 |
+
"lstrip": false,
|
| 63 |
+
"normalized": true,
|
| 64 |
+
"rstrip": false,
|
| 65 |
+
"single_word": false,
|
| 66 |
+
"special": false
|
| 67 |
+
},
|
| 68 |
+
"50264": {
|
| 69 |
+
"content": " ",
|
| 70 |
+
"lstrip": false,
|
| 71 |
+
"normalized": true,
|
| 72 |
+
"rstrip": false,
|
| 73 |
+
"single_word": false,
|
| 74 |
+
"special": false
|
| 75 |
+
},
|
| 76 |
+
"50265": {
|
| 77 |
+
"content": " ",
|
| 78 |
+
"lstrip": false,
|
| 79 |
+
"normalized": true,
|
| 80 |
+
"rstrip": false,
|
| 81 |
+
"single_word": false,
|
| 82 |
+
"special": false
|
| 83 |
+
},
|
| 84 |
+
"50266": {
|
| 85 |
+
"content": " ",
|
| 86 |
+
"lstrip": false,
|
| 87 |
+
"normalized": true,
|
| 88 |
+
"rstrip": false,
|
| 89 |
+
"single_word": false,
|
| 90 |
+
"special": false
|
| 91 |
+
},
|
| 92 |
+
"50267": {
|
| 93 |
+
"content": " ",
|
| 94 |
+
"lstrip": false,
|
| 95 |
+
"normalized": true,
|
| 96 |
+
"rstrip": false,
|
| 97 |
+
"single_word": false,
|
| 98 |
+
"special": false
|
| 99 |
+
},
|
| 100 |
+
"50268": {
|
| 101 |
+
"content": " ",
|
| 102 |
+
"lstrip": false,
|
| 103 |
+
"normalized": true,
|
| 104 |
+
"rstrip": false,
|
| 105 |
+
"single_word": false,
|
| 106 |
+
"special": false
|
| 107 |
+
},
|
| 108 |
+
"50269": {
|
| 109 |
+
"content": " ",
|
| 110 |
+
"lstrip": false,
|
| 111 |
+
"normalized": true,
|
| 112 |
+
"rstrip": false,
|
| 113 |
+
"single_word": false,
|
| 114 |
+
"special": false
|
| 115 |
+
},
|
| 116 |
+
"50270": {
|
| 117 |
+
"content": " ",
|
| 118 |
+
"lstrip": false,
|
| 119 |
+
"normalized": true,
|
| 120 |
+
"rstrip": false,
|
| 121 |
+
"single_word": false,
|
| 122 |
+
"special": false
|
| 123 |
+
},
|
| 124 |
+
"50271": {
|
| 125 |
+
"content": " ",
|
| 126 |
+
"lstrip": false,
|
| 127 |
+
"normalized": true,
|
| 128 |
+
"rstrip": false,
|
| 129 |
+
"single_word": false,
|
| 130 |
+
"special": false
|
| 131 |
+
},
|
| 132 |
+
"50272": {
|
| 133 |
+
"content": " ",
|
| 134 |
+
"lstrip": false,
|
| 135 |
+
"normalized": true,
|
| 136 |
+
"rstrip": false,
|
| 137 |
+
"single_word": false,
|
| 138 |
+
"special": false
|
| 139 |
+
},
|
| 140 |
+
"50273": {
|
| 141 |
+
"content": " ",
|
| 142 |
+
"lstrip": false,
|
| 143 |
+
"normalized": true,
|
| 144 |
+
"rstrip": false,
|
| 145 |
+
"single_word": false,
|
| 146 |
+
"special": false
|
| 147 |
+
},
|
| 148 |
+
"50274": {
|
| 149 |
+
"content": " ",
|
| 150 |
+
"lstrip": false,
|
| 151 |
+
"normalized": true,
|
| 152 |
+
"rstrip": false,
|
| 153 |
+
"single_word": false,
|
| 154 |
+
"special": false
|
| 155 |
+
},
|
| 156 |
+
"50275": {
|
| 157 |
+
"content": " ",
|
| 158 |
+
"lstrip": false,
|
| 159 |
+
"normalized": true,
|
| 160 |
+
"rstrip": false,
|
| 161 |
+
"single_word": false,
|
| 162 |
+
"special": false
|
| 163 |
+
},
|
| 164 |
+
"50276": {
|
| 165 |
+
"content": " ",
|
| 166 |
+
"lstrip": false,
|
| 167 |
+
"normalized": true,
|
| 168 |
+
"rstrip": false,
|
| 169 |
+
"single_word": false,
|
| 170 |
+
"special": false
|
| 171 |
+
},
|
| 172 |
+
"50277": {
|
| 173 |
+
"content": " ",
|
| 174 |
+
"lstrip": false,
|
| 175 |
+
"normalized": true,
|
| 176 |
+
"rstrip": false,
|
| 177 |
+
"single_word": false,
|
| 178 |
+
"special": false
|
| 179 |
+
},
|
| 180 |
+
"50278": {
|
| 181 |
+
"content": " ",
|
| 182 |
+
"lstrip": false,
|
| 183 |
+
"normalized": true,
|
| 184 |
+
"rstrip": false,
|
| 185 |
+
"single_word": false,
|
| 186 |
+
"special": false
|
| 187 |
+
},
|
| 188 |
+
"50279": {
|
| 189 |
+
"content": " ",
|
| 190 |
+
"lstrip": false,
|
| 191 |
+
"normalized": true,
|
| 192 |
+
"rstrip": false,
|
| 193 |
+
"single_word": false,
|
| 194 |
+
"special": false
|
| 195 |
+
},
|
| 196 |
+
"50280": {
|
| 197 |
+
"content": " ",
|
| 198 |
+
"lstrip": false,
|
| 199 |
+
"normalized": true,
|
| 200 |
+
"rstrip": false,
|
| 201 |
+
"single_word": false,
|
| 202 |
+
"special": false
|
| 203 |
+
},
|
| 204 |
+
"50281": {
|
| 205 |
+
"content": " ",
|
| 206 |
+
"lstrip": false,
|
| 207 |
+
"normalized": true,
|
| 208 |
+
"rstrip": false,
|
| 209 |
+
"single_word": false,
|
| 210 |
+
"special": false
|
| 211 |
+
},
|
| 212 |
+
"50282": {
|
| 213 |
+
"content": " ",
|
| 214 |
+
"lstrip": false,
|
| 215 |
+
"normalized": true,
|
| 216 |
+
"rstrip": false,
|
| 217 |
+
"single_word": false,
|
| 218 |
+
"special": false
|
| 219 |
+
},
|
| 220 |
+
"50283": {
|
| 221 |
+
"content": " ",
|
| 222 |
+
"lstrip": false,
|
| 223 |
+
"normalized": true,
|
| 224 |
+
"rstrip": false,
|
| 225 |
+
"single_word": false,
|
| 226 |
+
"special": false
|
| 227 |
+
},
|
| 228 |
+
"50284": {
|
| 229 |
+
"content": " ",
|
| 230 |
+
"lstrip": false,
|
| 231 |
+
"normalized": true,
|
| 232 |
+
"rstrip": false,
|
| 233 |
+
"single_word": false,
|
| 234 |
+
"special": false
|
| 235 |
+
},
|
| 236 |
+
"50285": {
|
| 237 |
+
"content": " ",
|
| 238 |
+
"lstrip": false,
|
| 239 |
+
"normalized": true,
|
| 240 |
+
"rstrip": false,
|
| 241 |
+
"single_word": false,
|
| 242 |
+
"special": false
|
| 243 |
+
},
|
| 244 |
+
"50286": {
|
| 245 |
+
"content": " ",
|
| 246 |
+
"lstrip": false,
|
| 247 |
+
"normalized": true,
|
| 248 |
+
"rstrip": false,
|
| 249 |
+
"single_word": false,
|
| 250 |
+
"special": false
|
| 251 |
+
},
|
| 252 |
+
"50287": {
|
| 253 |
+
"content": "\t\t\t\t\t\t\t\t\t",
|
| 254 |
+
"lstrip": false,
|
| 255 |
+
"normalized": true,
|
| 256 |
+
"rstrip": false,
|
| 257 |
+
"single_word": false,
|
| 258 |
+
"special": false
|
| 259 |
+
},
|
| 260 |
+
"50288": {
|
| 261 |
+
"content": "\t\t\t\t\t\t\t\t",
|
| 262 |
+
"lstrip": false,
|
| 263 |
+
"normalized": true,
|
| 264 |
+
"rstrip": false,
|
| 265 |
+
"single_word": false,
|
| 266 |
+
"special": false
|
| 267 |
+
},
|
| 268 |
+
"50289": {
|
| 269 |
+
"content": "\t\t\t\t\t\t\t",
|
| 270 |
+
"lstrip": false,
|
| 271 |
+
"normalized": true,
|
| 272 |
+
"rstrip": false,
|
| 273 |
+
"single_word": false,
|
| 274 |
+
"special": false
|
| 275 |
+
},
|
| 276 |
+
"50290": {
|
| 277 |
+
"content": "\t\t\t\t\t\t",
|
| 278 |
+
"lstrip": false,
|
| 279 |
+
"normalized": true,
|
| 280 |
+
"rstrip": false,
|
| 281 |
+
"single_word": false,
|
| 282 |
+
"special": false
|
| 283 |
+
},
|
| 284 |
+
"50291": {
|
| 285 |
+
"content": "\t\t\t\t\t",
|
| 286 |
+
"lstrip": false,
|
| 287 |
+
"normalized": true,
|
| 288 |
+
"rstrip": false,
|
| 289 |
+
"single_word": false,
|
| 290 |
+
"special": false
|
| 291 |
+
},
|
| 292 |
+
"50292": {
|
| 293 |
+
"content": "\t\t\t\t",
|
| 294 |
+
"lstrip": false,
|
| 295 |
+
"normalized": true,
|
| 296 |
+
"rstrip": false,
|
| 297 |
+
"single_word": false,
|
| 298 |
+
"special": false
|
| 299 |
+
},
|
| 300 |
+
"50293": {
|
| 301 |
+
"content": "\t\t\t",
|
| 302 |
+
"lstrip": false,
|
| 303 |
+
"normalized": true,
|
| 304 |
+
"rstrip": false,
|
| 305 |
+
"single_word": false,
|
| 306 |
+
"special": false
|
| 307 |
+
},
|
| 308 |
+
"50294": {
|
| 309 |
+
"content": "\t\t",
|
| 310 |
+
"lstrip": false,
|
| 311 |
+
"normalized": true,
|
| 312 |
+
"rstrip": false,
|
| 313 |
+
"single_word": false,
|
| 314 |
+
"special": false
|
| 315 |
+
}
|
| 316 |
+
},
|
| 317 |
+
"bos_token": "<|endoftext|>",
|
| 318 |
+
"clean_up_tokenization_spaces": true,
|
| 319 |
+
"eos_token": "<|endoftext|>",
|
| 320 |
+
"extra_special_tokens": {},
|
| 321 |
+
"model_max_length": 2048,
|
| 322 |
+
"return_token_type_ids": false,
|
| 323 |
+
"tokenizer_class": "CodeGenTokenizer",
|
| 324 |
+
"unk_token": "<|endoftext|>"
|
| 325 |
+
}
|
phi-2-lora-safecoder/checkpoint-last/vocab.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
phi-2-lora-safecoder/train.log
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
05/12/2026 02:48:32 - INFO - accelerate.utils.modeling - We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can set `max_memory` in to a higher value to use more memory (at your own risk).
|
| 2 |
+
05/12/2026 02:48:53 - INFO - root - number of sec samples before upsampling: 1809
|
| 3 |
+
05/12/2026 02:48:53 - INFO - root - number of sec samples after upsampling: 3344
|
| 4 |
+
05/12/2026 02:48:56 - INFO - root - Training args Namespace(output_name='phi-2-lora-safecoder', datasets=['lmsys', 'sec-desc', 'sec-new-desc'], pretrain_name='phi-2', loss_weight=1.0, sven=False, num_train_epochs=2, learning_rate=2e-05, max_num_tokens=1024, batch_size=1, grad_acc_steps=16, weight_decay=0.01, adam_epsilon=1e-08, warmup_steps=0, max_grad_norm=1.0, dropout=0.1, kl_loss_weight=0, exclude_neg=False, no_weights=False, lora=True, r=16, lora_alpha=32, lora_dropout=0.1, sampling_size=40, sampling_method='minority', cwes=['all'], langs=['all'], logging_steps=50, save_epochs=10, seed=2, data_dir='../data_train_val', model_dir='../trained/', output_dir='../trained/phi-2-lora-safecoder', logger=<RootLogger root (INFO)>)
|
| 5 |
+
05/12/2026 02:48:56 - INFO - root - ***** Running training *****
|
| 6 |
+
05/12/2026 02:48:56 - INFO - root - Num samples = 19626
|
| 7 |
+
05/12/2026 02:48:56 - INFO - root - Num epoch = 2
|
| 8 |
+
05/12/2026 02:48:56 - INFO - root - Batch size= 1
|
| 9 |
+
05/12/2026 02:48:56 - INFO - root - Total batch size (w. accumulation) = 16
|
| 10 |
+
05/12/2026 02:48:56 - INFO - root - Gradient Accumulation steps = 16
|
| 11 |
+
05/12/2026 02:48:56 - INFO - root - Total optimization steps = 2452
|
| 12 |
+
05/12/2026 02:48:56 - INFO - root - Num val samples = 2008
|
| 13 |
+
05/12/2026 02:48:56 - INFO - root - Num parameters = 2799487975
|
| 14 |
+
05/12/2026 02:48:56 - INFO - root - Num trainable parameters = 24438640
|
| 15 |
+
05/12/2026 02:52:58 - INFO - root - epochs: 1/2, steps: 50/2452, func: 0.087809, pos: 0.115657, neg: 0.123793, 1%: 3h 14m 29s
|
| 16 |
+
05/12/2026 02:56:56 - INFO - root - epochs: 1/2, steps: 100/2452, func: 0.078225, pos: 0.093975, neg: 0.098627, 4%: 3h 8m 16s
|
| 17 |
+
05/12/2026 03:00:50 - INFO - root - epochs: 1/2, steps: 150/2452, func: 0.073382, pos: 0.100339, neg: 0.069674, 6%: 3h 2m 54s
|
| 18 |
+
05/12/2026 03:04:47 - INFO - root - epochs: 1/2, steps: 200/2452, func: 0.072408, pos: 0.092015, neg: 0.061007, 8%: 2h 58m 33s
|
| 19 |
+
05/12/2026 03:08:39 - INFO - root - epochs: 1/2, steps: 250/2452, func: 0.069396, pos: 0.097156, neg: 0.060685, 10%: 2h 53m 41s
|
| 20 |
+
05/12/2026 03:12:32 - INFO - root - epochs: 1/2, steps: 300/2452, func: 0.070725, pos: 0.099053, neg: 0.050802, 12%: 2h 49m 18s
|
| 21 |
+
05/12/2026 03:16:21 - INFO - root - epochs: 1/2, steps: 350/2452, func: 0.069297, pos: 0.092138, neg: 0.050556, 14%: 2h 44m 48s
|
| 22 |
+
05/12/2026 03:20:14 - INFO - root - epochs: 1/2, steps: 400/2452, func: 0.069739, pos: 0.093173, neg: 0.051642, 16%: 2h 40m 37s
|
| 23 |
+
05/12/2026 03:24:01 - INFO - root - epochs: 1/2, steps: 450/2452, func: 0.06755, pos: 0.090788, neg: 0.049833, 18%: 2h 36m 11s
|
| 24 |
+
05/12/2026 03:27:55 - INFO - root - epochs: 1/2, steps: 500/2452, func: 0.067713, pos: 0.092051, neg: 0.044417, 20%: 2h 32m 16s
|
| 25 |
+
05/12/2026 03:31:38 - INFO - root - epochs: 1/2, steps: 550/2452, func: 0.067988, pos: 0.075132, neg: 0.03882, 22%: 2h 27m 45s
|
| 26 |
+
05/12/2026 03:35:33 - INFO - root - epochs: 1/2, steps: 600/2452, func: 0.064584, pos: 0.083783, neg: 0.03723, 24%: 2h 23m 58s
|
| 27 |
+
05/12/2026 03:39:25 - INFO - root - epochs: 1/2, steps: 650/2452, func: 0.066767, pos: 0.08717, neg: 0.029386, 26%: 2h 20m 1s
|
| 28 |
+
05/12/2026 03:43:18 - INFO - root - epochs: 1/2, steps: 700/2452, func: 0.068934, pos: 0.107346, neg: 0.033175, 28%: 2h 16m 10s
|
| 29 |
+
05/12/2026 03:47:08 - INFO - root - epochs: 1/2, steps: 750/2452, func: 0.066618, pos: 0.081592, neg: 0.033057, 30%: 2h 12m 10s
|
| 30 |
+
05/12/2026 03:50:56 - INFO - root - epochs: 1/2, steps: 800/2452, func: 0.064837, pos: 0.086676, neg: 0.028554, 32%: 2h 8m 5s
|
| 31 |
+
05/12/2026 03:54:42 - INFO - root - epochs: 1/2, steps: 850/2452, func: 0.064665, pos: 0.082752, neg: 0.035432, 34%: 2h 4m 2s
|
| 32 |
+
05/12/2026 03:58:34 - INFO - root - epochs: 1/2, steps: 900/2452, func: 0.064953, pos: 0.079248, neg: 0.030881, 36%: 2h 0m 8s
|
| 33 |
+
05/12/2026 04:02:29 - INFO - root - epochs: 1/2, steps: 950/2452, func: 0.065705, pos: 0.075704, neg: 0.02721, 38%: 1h 56m 21s
|
| 34 |
+
05/12/2026 04:06:28 - INFO - root - epochs: 1/2, steps: 1000/2452, func: 0.06857, pos: 0.082659, neg: 0.033367, 40%: 1h 52m 39s
|
| 35 |
+
05/12/2026 04:10:28 - INFO - root - epochs: 1/2, steps: 1050/2452, func: 0.066857, pos: 0.071119, neg: 0.032058, 42%: 1h 48m 57s
|
| 36 |
+
05/12/2026 04:14:21 - INFO - root - epochs: 1/2, steps: 1100/2452, func: 0.065121, pos: 0.072074, neg: 0.034161, 44%: 1h 45m 4s
|
| 37 |
+
05/12/2026 04:18:13 - INFO - root - epochs: 1/2, steps: 1150/2452, func: 0.067815, pos: 0.064138, neg: 0.02986, 46%: 1h 41m 10s
|
| 38 |
+
05/12/2026 04:22:04 - INFO - root - epochs: 1/2, steps: 1200/2452, func: 0.065764, pos: 0.064906, neg: 0.028599, 48%: 1h 37m 14s
|
| 39 |
+
05/12/2026 04:26:05 - INFO - root - epochs: 2/2, steps: 1250/2452, func: 0.064512, pos: 0.077691, neg: 0.032867, 50%: 1h 33m 30s
|
| 40 |
+
05/12/2026 04:29:58 - INFO - root - epochs: 2/2, steps: 1300/2452, func: 0.064248, pos: 0.082205, neg: 0.026269, 52%: 1h 29m 36s
|
| 41 |
+
05/12/2026 04:33:54 - INFO - root - epochs: 2/2, steps: 1350/2452, func: 0.066843, pos: 0.063788, neg: 0.029562, 55%: 1h 25m 45s
|
| 42 |
+
05/12/2026 04:37:47 - INFO - root - epochs: 2/2, steps: 1400/2452, func: 0.061568, pos: 0.058695, neg: 0.032394, 57%: 1h 21m 52s
|
| 43 |
+
05/12/2026 04:41:41 - INFO - root - epochs: 2/2, steps: 1450/2452, func: 0.061593, pos: 0.070217, neg: 0.023268, 59%: 1h 17m 58s
|
| 44 |
+
05/12/2026 04:45:39 - INFO - root - epochs: 2/2, steps: 1500/2452, func: 0.063944, pos: 0.072002, neg: 0.030755, 61%: 1h 14m 9s
|
| 45 |
+
05/12/2026 04:49:32 - INFO - root - epochs: 2/2, steps: 1550/2452, func: 0.063425, pos: 0.050255, neg: 0.027983, 63%: 1h 10m 15s
|
| 46 |
+
05/12/2026 04:53:25 - INFO - root - epochs: 2/2, steps: 1600/2452, func: 0.062742, pos: 0.073932, neg: 0.022697, 65%: 1h 6m 21s
|
| 47 |
+
05/12/2026 04:57:15 - INFO - root - epochs: 2/2, steps: 1650/2452, func: 0.063407, pos: 0.065383, neg: 0.029697, 67%: 1h 2m 27s
|
| 48 |
+
05/12/2026 05:01:06 - INFO - root - epochs: 2/2, steps: 1700/2452, func: 0.063459, pos: 0.065671, neg: 0.020674, 69%: 0h 58m 33s
|
| 49 |
+
05/12/2026 05:04:58 - INFO - root - epochs: 2/2, steps: 1750/2452, func: 0.064214, pos: 0.058392, neg: 0.021934, 71%: 0h 54m 38s
|
| 50 |
+
05/12/2026 05:08:42 - INFO - root - epochs: 2/2, steps: 1800/2452, func: 0.064039, pos: 0.076992, neg: 0.02203, 73%: 0h 50m 42s
|
| 51 |
+
05/12/2026 05:12:39 - INFO - root - epochs: 2/2, steps: 1850/2452, func: 0.063196, pos: 0.049055, neg: 0.026583, 75%: 0h 46m 50s
|
| 52 |
+
05/12/2026 05:16:32 - INFO - root - epochs: 2/2, steps: 1900/2452, func: 0.063654, pos: 0.059067, neg: 0.024511, 77%: 0h 42m 57s
|
| 53 |
+
05/12/2026 05:20:23 - INFO - root - epochs: 2/2, steps: 1950/2452, func: 0.063591, pos: 0.066934, neg: 0.022828, 79%: 0h 39m 4s
|
| 54 |
+
05/12/2026 05:24:17 - INFO - root - epochs: 2/2, steps: 2000/2452, func: 0.063182, pos: 0.078367, neg: 0.028742, 81%: 0h 35m 11s
|
| 55 |
+
05/12/2026 05:28:02 - INFO - root - epochs: 2/2, steps: 2050/2452, func: 0.061853, pos: 0.071412, neg: 0.026568, 83%: 0h 31m 16s
|
| 56 |
+
05/12/2026 05:31:53 - INFO - root - epochs: 2/2, steps: 2100/2452, func: 0.062466, pos: 0.064775, neg: 0.03288, 85%: 0h 27m 23s
|
| 57 |
+
05/12/2026 05:35:51 - INFO - root - epochs: 2/2, steps: 2150/2452, func: 0.066738, pos: 0.050037, neg: 0.019877, 87%: 0h 23m 31s
|
| 58 |
+
05/12/2026 05:39:47 - INFO - root - epochs: 2/2, steps: 2200/2452, func: 0.064148, pos: 0.061206, neg: 0.027983, 89%: 0h 19m 38s
|
| 59 |
+
05/12/2026 05:43:39 - INFO - root - epochs: 2/2, steps: 2250/2452, func: 0.064126, pos: 0.06231, neg: 0.027208, 91%: 0h 15m 45s
|
| 60 |
+
05/12/2026 05:47:34 - INFO - root - epochs: 2/2, steps: 2300/2452, func: 0.062785, pos: 0.081734, neg: 0.021968, 93%: 0h 11m 53s
|
| 61 |
+
05/12/2026 05:51:27 - INFO - root - epochs: 2/2, steps: 2350/2452, func: 0.063474, pos: 0.065291, neg: 0.030004, 95%: 0h 8m 0s
|
| 62 |
+
05/12/2026 05:55:28 - INFO - root - epochs: 2/2, steps: 2400/2452, func: 0.064366, pos: 0.063885, neg: 0.02885, 97%: 0h 4m 7s
|
| 63 |
+
05/12/2026 05:59:30 - INFO - root - epochs: 2/2, steps: 2450/2452, func: 0.065632, pos: 0.059616, neg: 0.02787, 99%: 0h 0m 14s
|
| 64 |
+
05/12/2026 06:04:10 - INFO - root - final eval loss: func: 0.062987, pos: 0.079249, neg: 0.036373
|
| 65 |
+
05/12/2026 06:04:10 - INFO - root - Saving model checkpoint to ../trained/phi-2-lora-safecoder/checkpoint-last
|
qwen2.5-coder-3b-lora-safecoder/checkpoint-last/README.md
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: Qwen/Qwen2.5-Coder-3B-Instruct
|
| 3 |
+
library_name: peft
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Model Card for Model ID
|
| 7 |
+
|
| 8 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
## Model Details
|
| 13 |
+
|
| 14 |
+
### Model Description
|
| 15 |
+
|
| 16 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
- **Developed by:** [More Information Needed]
|
| 21 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 22 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 23 |
+
- **Model type:** [More Information Needed]
|
| 24 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 25 |
+
- **License:** [More Information Needed]
|
| 26 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 27 |
+
|
| 28 |
+
### Model Sources [optional]
|
| 29 |
+
|
| 30 |
+
<!-- Provide the basic links for the model. -->
|
| 31 |
+
|
| 32 |
+
- **Repository:** [More Information Needed]
|
| 33 |
+
- **Paper [optional]:** [More Information Needed]
|
| 34 |
+
- **Demo [optional]:** [More Information Needed]
|
| 35 |
+
|
| 36 |
+
## Uses
|
| 37 |
+
|
| 38 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 39 |
+
|
| 40 |
+
### Direct Use
|
| 41 |
+
|
| 42 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 43 |
+
|
| 44 |
+
[More Information Needed]
|
| 45 |
+
|
| 46 |
+
### Downstream Use [optional]
|
| 47 |
+
|
| 48 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 49 |
+
|
| 50 |
+
[More Information Needed]
|
| 51 |
+
|
| 52 |
+
### Out-of-Scope Use
|
| 53 |
+
|
| 54 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 55 |
+
|
| 56 |
+
[More Information Needed]
|
| 57 |
+
|
| 58 |
+
## Bias, Risks, and Limitations
|
| 59 |
+
|
| 60 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 61 |
+
|
| 62 |
+
[More Information Needed]
|
| 63 |
+
|
| 64 |
+
### Recommendations
|
| 65 |
+
|
| 66 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 67 |
+
|
| 68 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 69 |
+
|
| 70 |
+
## How to Get Started with the Model
|
| 71 |
+
|
| 72 |
+
Use the code below to get started with the model.
|
| 73 |
+
|
| 74 |
+
[More Information Needed]
|
| 75 |
+
|
| 76 |
+
## Training Details
|
| 77 |
+
|
| 78 |
+
### Training Data
|
| 79 |
+
|
| 80 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 81 |
+
|
| 82 |
+
[More Information Needed]
|
| 83 |
+
|
| 84 |
+
### Training Procedure
|
| 85 |
+
|
| 86 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 87 |
+
|
| 88 |
+
#### Preprocessing [optional]
|
| 89 |
+
|
| 90 |
+
[More Information Needed]
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
#### Training Hyperparameters
|
| 94 |
+
|
| 95 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 96 |
+
|
| 97 |
+
#### Speeds, Sizes, Times [optional]
|
| 98 |
+
|
| 99 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 100 |
+
|
| 101 |
+
[More Information Needed]
|
| 102 |
+
|
| 103 |
+
## Evaluation
|
| 104 |
+
|
| 105 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 106 |
+
|
| 107 |
+
### Testing Data, Factors & Metrics
|
| 108 |
+
|
| 109 |
+
#### Testing Data
|
| 110 |
+
|
| 111 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 112 |
+
|
| 113 |
+
[More Information Needed]
|
| 114 |
+
|
| 115 |
+
#### Factors
|
| 116 |
+
|
| 117 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 118 |
+
|
| 119 |
+
[More Information Needed]
|
| 120 |
+
|
| 121 |
+
#### Metrics
|
| 122 |
+
|
| 123 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 124 |
+
|
| 125 |
+
[More Information Needed]
|
| 126 |
+
|
| 127 |
+
### Results
|
| 128 |
+
|
| 129 |
+
[More Information Needed]
|
| 130 |
+
|
| 131 |
+
#### Summary
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
## Model Examination [optional]
|
| 136 |
+
|
| 137 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 138 |
+
|
| 139 |
+
[More Information Needed]
|
| 140 |
+
|
| 141 |
+
## Environmental Impact
|
| 142 |
+
|
| 143 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 144 |
+
|
| 145 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 146 |
+
|
| 147 |
+
- **Hardware Type:** [More Information Needed]
|
| 148 |
+
- **Hours used:** [More Information Needed]
|
| 149 |
+
- **Cloud Provider:** [More Information Needed]
|
| 150 |
+
- **Compute Region:** [More Information Needed]
|
| 151 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 152 |
+
|
| 153 |
+
## Technical Specifications [optional]
|
| 154 |
+
|
| 155 |
+
### Model Architecture and Objective
|
| 156 |
+
|
| 157 |
+
[More Information Needed]
|
| 158 |
+
|
| 159 |
+
### Compute Infrastructure
|
| 160 |
+
|
| 161 |
+
[More Information Needed]
|
| 162 |
+
|
| 163 |
+
#### Hardware
|
| 164 |
+
|
| 165 |
+
[More Information Needed]
|
| 166 |
+
|
| 167 |
+
#### Software
|
| 168 |
+
|
| 169 |
+
[More Information Needed]
|
| 170 |
+
|
| 171 |
+
## Citation [optional]
|
| 172 |
+
|
| 173 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 174 |
+
|
| 175 |
+
**BibTeX:**
|
| 176 |
+
|
| 177 |
+
[More Information Needed]
|
| 178 |
+
|
| 179 |
+
**APA:**
|
| 180 |
+
|
| 181 |
+
[More Information Needed]
|
| 182 |
+
|
| 183 |
+
## Glossary [optional]
|
| 184 |
+
|
| 185 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 186 |
+
|
| 187 |
+
[More Information Needed]
|
| 188 |
+
|
| 189 |
+
## More Information [optional]
|
| 190 |
+
|
| 191 |
+
[More Information Needed]
|
| 192 |
+
|
| 193 |
+
## Model Card Authors [optional]
|
| 194 |
+
|
| 195 |
+
[More Information Needed]
|
| 196 |
+
|
| 197 |
+
## Model Card Contact
|
| 198 |
+
|
| 199 |
+
[More Information Needed]
|
| 200 |
+
### Framework versions
|
| 201 |
+
|
| 202 |
+
- PEFT 0.8.2
|