alenphilip commited on
Commit
5459acf
·
verified ·
1 Parent(s): 8efb93a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -17
README.md CHANGED
@@ -198,28 +198,28 @@ The model was trained on a comprehensive dataset of Python code review examples
198
  ## Training Procedure
199
  [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/alenphilip2071-google/huggingface/runs/d27nrifd)
200
  ### Training Hyperparameters
201
- - Training regime: bf16 mixed precision with SFT & QLoRA
202
- - Base Model: Qwen2.5-7B-Instruct
203
- - LoRA Rank: 32
204
- - LoRA Alpha: 64
205
- - LoRA Dropout: 0.1
206
- - Learning Rate: 2e-4
207
- - Batch Size: 16 (with gradient accumulation 4)
208
- - Epochs: 2
209
- - Max Sequence Length: 2048 tokens
210
- - Optimizer: Paged AdamW 8-bit
211
 
212
  ### Speeds, Sizes, Times
213
- - Base Model Size: 7B parameters
214
- - Adapter Size: ~45MB
215
- - Training Time: ~68 minutes for 400 steps
216
- - Training Examples: 13,670 training, 1,726 evaluation
217
 
218
  ## Evaluation
219
  ### Metrics
220
- ROUGE-L: 0.754
221
- BLEU: 61.99
222
- Validation Loss: 0.595
223
 
224
  ## Results
225
  The model achieved strong performance on code review tasks, particularly excelling at:
 
198
  ## Training Procedure
199
  [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/alenphilip2071-google/huggingface/runs/d27nrifd)
200
  ### Training Hyperparameters
201
+ - **Training regime:** bf16 mixed precision with SFT & QLoRA
202
+ - **Base Model:** Qwen2.5-7B-Instruct
203
+ - **LoRA Rank:** 32
204
+ - **LoRA Alpha:** 64
205
+ - **LoRA Dropout:** 0.1
206
+ - **Learning Rate:** 2e-4
207
+ - **Batch Size:** 16 (with gradient accumulation 4)
208
+ - **Epochs:** 2
209
+ - **Max Sequence Length:** 2048 tokens
210
+ - **Optimizer:** Paged AdamW 8-bit
211
 
212
  ### Speeds, Sizes, Times
213
+ - **Base Model Size:** 7B parameters
214
+ - **Adapter Size:** ~45MB
215
+ - **Training Time:** ~68 minutes for 400 steps
216
+ - **Training Examples:** 13,670 training, 1,726 evaluation
217
 
218
  ## Evaluation
219
  ### Metrics
220
+ - **ROUGE-L:** 0.754
221
+ - **BLEU:** 61.99
222
+ - **Validation Loss:** 0.595
223
 
224
  ## Results
225
  The model achieved strong performance on code review tasks, particularly excelling at: