Spaces:

---

HermitQ

/

NPCAlign-Demo

like 0

Sleeping

NPCAlign — NPC Quest Dialogue Fine-tuning Demo

This is a personal project that fine-tunes Llama-3.1-8B-Instruct for RPG NPC quest dialogue generation using a two-stage SFT → DPO alignment pipeline. The demo enables blind side-by-side human evaluation of the SFT and DPO models.

Why GPU is needed: The demo loads two LoRA adapters (SFT and DPO) on top of Llama-3.1-8B-Instruct. With ZeroGPU, each visitor's personal quota is exhausted after just a few conversation turns (ZeroGPU charges per reserved time block, not actual usage), making it impossible to complete a full evaluation session. A persistent GPU would allow evaluators to complete the full 6-step evaluation flow (NPC config → chat × 2 → rate × 2 → submit) without interruption.

Project details:
Base model: meta-llama/Meta-Llama-3.1-8B-Instruct
SFT adapter: HermitQ/NPCAlign-SFT
DPO adapter: HermitQ/NPCAlign-DPO
Training data: chimbiwide/NPC-Quest-Dialogue (open dataset)
Evaluation data is saved to a Storage Bucket for analysis
GitHub: https://github.com/Hermit888/NPCAlign