benhs000 commited on
Commit
3391eb6
·
verified ·
1 Parent(s): b83672b

added Issues to be addressed 🚧

Browse files
Files changed (1) hide show
  1. README.md +6 -3
README.md CHANGED
@@ -43,14 +43,13 @@ This is especially tuned for *game developers* who want believable character dia
43
 
44
  | Aspect | Description |
45
  |--------|--------------|
46
- | **Base Model** | Qwen3-4B-Instruct (Apache 2.0) |
47
  | **Method** | Unsloth + TRL LoRA fine-tuning |
48
  | **LoRA Config** | r=16, alpha=16, 1 epoch, lr=2e-4 |
49
  | **Dataset** | ~10k RP dialogues: branching quests, adaptive NPCs, synthetic "memory" cues |
50
- | **Special Token** | `/nothink` for immediate, non-reasoned responses |
51
  | **Hardware** | Single GPU (T4), 20-minute training |
52
  | **Quantization** | GGUF Q4_K_M (~2.1GB) for CPU & M1 use |
53
- | **Eval Summary** | 12% perplexity drop on RP benchmarks; context-aware, non-repetitive NPCs |
54
 
55
  ---
56
 
@@ -168,6 +167,10 @@ Output:
168
 
169
  ---
170
 
 
 
 
 
171
  ## 📚 Citation
172
 
173
  Schneider, B. (2025). *EmergentRP-Qwen4B* [Fine-tuned model]. Hugging Face.
 
43
 
44
  | Aspect | Description |
45
  |--------|--------------|
46
+ | **Base Model** | Qwen/Qwen3-4B-Instruct-2507 (Apache 2.0) |
47
  | **Method** | Unsloth + TRL LoRA fine-tuning |
48
  | **LoRA Config** | r=16, alpha=16, 1 epoch, lr=2e-4 |
49
  | **Dataset** | ~10k RP dialogues: branching quests, adaptive NPCs, synthetic "memory" cues |
 
50
  | **Hardware** | Single GPU (T4), 20-minute training |
51
  | **Quantization** | GGUF Q4_K_M (~2.1GB) for CPU & M1 use |
52
+ | **Eval Summary** | 12% perplexity drop on RP benchmarks; context-aware, non-repetitive NPCs (still in progress) |
53
 
54
  ---
55
 
 
167
 
168
  ---
169
 
170
+ ## 🚧 Found Issues to be addressed
171
+ - Sometimes the model mentions that it's not able to role-play which likely comes in from the quantization and limited fine-tunes.
172
+ - With pre-existing contexts the model can enter an endless repetition loop -> perhaps adjusting my trainings data-sets to capture these systematically will help.
173
+
174
  ## 📚 Citation
175
 
176
  Schneider, B. (2025). *EmergentRP-Qwen4B* [Fine-tuned model]. Hugging Face.