Hugging Face's logo

LoftQ
/
phi-2-4bit-64rank

Text Generation
Transformers
Safetensors
English
phi
quantization
lora
custom_code
text-generation-inference
Model card Files
xet
 Community
LoftQ commited on
Commit
139821d
·
verified ·
1 Parent(s): 8685d03

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -64,8 +64,8 @@ We have conducted experiments on supervised fine-tuning of [GSM8K](https://huggi
64
 
65
  | Model | Bits | Rank | LoRA Initial | GSM8K |
66
  | --------| ---- | ---- | ---------------------- | --------- |
67
- | Phi-2 | 16 | 64 | Full model fine-tuning | 66.8±1.2 |
68
- | Phi-2 | 16 | 64 | Gaussian + 0 | 64.8±0.5 |
69
  | Phi-2 | 4 | 64 | Gaussian + 0 (QLoRA) | 60.2±0.6 |
70
  | Phi-2 | 4 | 64 | LoftQ | 64.1±0.7 |
71
 
 
64
 
65
  | Model | Bits | Rank | LoRA Initial | GSM8K |
66
  | --------| ---- | ---- | ---------------------- | --------- |
67
+ | Phi-2 | 16 | - | Full model fine-tuning | 66.8±1.2 |
68
+ | Phi-2 | 16 | 64 | Gaussian + 0 (LoRA) | 64.8±0.5 |
69
  | Phi-2 | 4 | 64 | Gaussian + 0 (QLoRA) | 60.2±0.6 |
70
  | Phi-2 | 4 | 64 | LoftQ | 64.1±0.7 |
71