Update root README with Qwen3_4B_Full_1k model info
Browse files
README.md
CHANGED
|
@@ -18,9 +18,19 @@ This repository contains various fine-tuned models for negotiation message recon
|
|
| 18 |
- **Method**: LoRA (Low-Rank Adaptation)
|
| 19 |
- **Training samples**: ~1k negotiation examples
|
| 20 |
- **Hardware**: 2x A100 GPUs
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
## Usage
|
| 23 |
|
|
|
|
| 24 |
```python
|
| 25 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 26 |
from peft import PeftModel
|
|
@@ -32,3 +42,19 @@ tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-4B-Instruct")
|
|
| 32 |
# Load LoRA adapter
|
| 33 |
model = PeftModel.from_pretrained(base_model, "JLiangHe/negotiation_clone", subfolder="Qwen3_4B_LORA_1k")
|
| 34 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
- **Method**: LoRA (Low-Rank Adaptation)
|
| 19 |
- **Training samples**: ~1k negotiation examples
|
| 20 |
- **Hardware**: 2x A100 GPUs
|
| 21 |
+
- **Size**: ~505MB (LoRA adapter only)
|
| 22 |
+
|
| 23 |
+
### Qwen3_4B_Full_1k
|
| 24 |
+
- **Location**: `Qwen3_4B_Full_1k/`
|
| 25 |
+
- **Base model**: Qwen/Qwen2.5-4B-Instruct
|
| 26 |
+
- **Method**: Full fine-tuning (all parameters)
|
| 27 |
+
- **Training samples**: ~1k negotiation examples
|
| 28 |
+
- **Hardware**: 2x A100 GPUs with DeepSpeed ZeRO-3
|
| 29 |
+
- **Size**: ~7.6GB (full model weights)
|
| 30 |
|
| 31 |
## Usage
|
| 32 |
|
| 33 |
+
### Using LoRA model
|
| 34 |
```python
|
| 35 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 36 |
from peft import PeftModel
|
|
|
|
| 42 |
# Load LoRA adapter
|
| 43 |
model = PeftModel.from_pretrained(base_model, "JLiangHe/negotiation_clone", subfolder="Qwen3_4B_LORA_1k")
|
| 44 |
```
|
| 45 |
+
|
| 46 |
+
### Using Full fine-tuned model
|
| 47 |
+
```python
|
| 48 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 49 |
+
|
| 50 |
+
# Load full fine-tuned model directly
|
| 51 |
+
model = AutoModelForCausalLM.from_pretrained("JLiangHe/negotiation_clone", subfolder="Qwen3_4B_Full_1k")
|
| 52 |
+
tokenizer = AutoTokenizer.from_pretrained("JLiangHe/negotiation_clone", subfolder="Qwen3_4B_Full_1k")
|
| 53 |
+
```
|
| 54 |
+
|
| 55 |
+
## Model Comparison
|
| 56 |
+
|
| 57 |
+
| Model | Method | Size | Memory | Performance |
|
| 58 |
+
|-------|--------|------|--------|-------------|
|
| 59 |
+
| Qwen3_4B_LORA_1k | LoRA | 505MB | Low (base + adapter) | Good for inference efficiency |
|
| 60 |
+
| Qwen3_4B_Full_1k | Full FT | 7.6GB | High (full model) | May have better task adaptation |
|