Update README.md
Browse files
README.md
CHANGED
|
@@ -27,7 +27,9 @@ This project fine-tunes a language model using supervised fine-tuning (SFT) and
|
|
| 27 |
For optimal performance, **chunk your text** into smaller, coherent pieces before providing it to the model. Long documents can lead the model to focus on specific details rather than the overall context.
|
| 28 |
|
| 29 |
- **Training Setup:**
|
| 30 |
-
The model is fine-tuned using the Unsloth framework with LoRA adapters, taking advantage of an A100 GPU for efficient training.
|
|
|
|
|
|
|
| 31 |
|
| 32 |
## Quick Usage
|
| 33 |
|
|
@@ -70,6 +72,7 @@ _ = model.generate(
|
|
| 70 |
min_p=0.1,
|
| 71 |
eos_token_id=tokenizer.eos_token_id, # Ensures proper termination.
|
| 72 |
)
|
|
|
|
| 73 |
|
| 74 |
# Uploaded model
|
| 75 |
|
|
|
|
| 27 |
For optimal performance, **chunk your text** into smaller, coherent pieces before providing it to the model. Long documents can lead the model to focus on specific details rather than the overall context.
|
| 28 |
|
| 29 |
- **Training Setup:**
|
| 30 |
+
The model is fine-tuned using the Unsloth framework with LoRA adapters, taking advantage of an A100 GPU for efficient training. See W&B loss curve here: https://wandb.ai/prdev/lora_model_training/panel/jp2r24xk7?nw=nwuserprdev
|
| 31 |
+
|
| 32 |
+
|
| 33 |
|
| 34 |
## Quick Usage
|
| 35 |
|
|
|
|
| 72 |
min_p=0.1,
|
| 73 |
eos_token_id=tokenizer.eos_token_id, # Ensures proper termination.
|
| 74 |
)
|
| 75 |
+
```
|
| 76 |
|
| 77 |
# Uploaded model
|
| 78 |
|