added Issues to be addressed 🚧
Browse files
README.md
CHANGED
|
@@ -43,14 +43,13 @@ This is especially tuned for *game developers* who want believable character dia
|
|
| 43 |
|
| 44 |
| Aspect | Description |
|
| 45 |
|--------|--------------|
|
| 46 |
-
| **Base Model** | Qwen3-4B-Instruct (Apache 2.0) |
|
| 47 |
| **Method** | Unsloth + TRL LoRA fine-tuning |
|
| 48 |
| **LoRA Config** | r=16, alpha=16, 1 epoch, lr=2e-4 |
|
| 49 |
| **Dataset** | ~10k RP dialogues: branching quests, adaptive NPCs, synthetic "memory" cues |
|
| 50 |
-
| **Special Token** | `/nothink` for immediate, non-reasoned responses |
|
| 51 |
| **Hardware** | Single GPU (T4), 20-minute training |
|
| 52 |
| **Quantization** | GGUF Q4_K_M (~2.1GB) for CPU & M1 use |
|
| 53 |
-
| **Eval Summary** | 12% perplexity drop on RP benchmarks; context-aware, non-repetitive NPCs |
|
| 54 |
|
| 55 |
---
|
| 56 |
|
|
@@ -168,6 +167,10 @@ Output:
|
|
| 168 |
|
| 169 |
---
|
| 170 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 171 |
## 📚 Citation
|
| 172 |
|
| 173 |
Schneider, B. (2025). *EmergentRP-Qwen4B* [Fine-tuned model]. Hugging Face.
|
|
|
|
| 43 |
|
| 44 |
| Aspect | Description |
|
| 45 |
|--------|--------------|
|
| 46 |
+
| **Base Model** | Qwen/Qwen3-4B-Instruct-2507 (Apache 2.0) |
|
| 47 |
| **Method** | Unsloth + TRL LoRA fine-tuning |
|
| 48 |
| **LoRA Config** | r=16, alpha=16, 1 epoch, lr=2e-4 |
|
| 49 |
| **Dataset** | ~10k RP dialogues: branching quests, adaptive NPCs, synthetic "memory" cues |
|
|
|
|
| 50 |
| **Hardware** | Single GPU (T4), 20-minute training |
|
| 51 |
| **Quantization** | GGUF Q4_K_M (~2.1GB) for CPU & M1 use |
|
| 52 |
+
| **Eval Summary** | 12% perplexity drop on RP benchmarks; context-aware, non-repetitive NPCs (still in progress) |
|
| 53 |
|
| 54 |
---
|
| 55 |
|
|
|
|
| 167 |
|
| 168 |
---
|
| 169 |
|
| 170 |
+
## 🚧 Found Issues to be addressed
|
| 171 |
+
- Sometimes the model mentions that it's not able to role-play which likely comes in from the quantization and limited fine-tunes.
|
| 172 |
+
- With pre-existing contexts the model can enter an endless repetition loop -> perhaps adjusting my trainings data-sets to capture these systematically will help.
|
| 173 |
+
|
| 174 |
## 📚 Citation
|
| 175 |
|
| 176 |
Schneider, B. (2025). *EmergentRP-Qwen4B* [Fine-tuned model]. Hugging Face.
|