clemencetran/questcrafter-finetuned

QuestCrafter – Fine-Tuned DistilGPT2 for RPG Quest Generation

Model Overview

• Base model: DistilGPT2

• Fine-tuned on: TinyStories (100,000 sampled examples)

• Task: Controllable fantasy quest generation

• Control tokens: , , ,

• QuestCrafter is a fine-tuned DistilGPT2 model designed to generate structured fantasy RPG quests conditioned on explicit control tokens.This project was developed as part of the MSc AI Machine Learning course.

Intended Use

This model is intended for:

• Generating short fantasy quest stories
• Educational demonstrations of controllable text generation
• NLP coursework and research experiments
• Prototype RPG quest systems

This model is not intended for:

• High-stakes decision making
• Production deployment without safeguards
• Safety-critical applications

Training Procedure

The model was fine-tuned using the Hugging Face Trainer API with:

• 3 training epochs
• Batch size: 4
• Learning rate: 5e-5
• Maximum sequence length: 256 tokens
• Causal language modeling objective

Evaluation

Automatic Evaluation Results

Model	Validation Loss	Perplexity	Distinct-1	Distinct-2
Baseline (DistilGPT2)	3.1764	23.96	0.2708	0.7762
Fine-Tuned	1.4080	4.09	0.1528	0.5140

=> The fine-tuned model greatly reduces perplexity compared to the baseline, showing improved coherence and dataset alignment. Although diversity slightly decreases, the generated quests are more structured and consistent.

Human Evaluation Results

Model	Coherence (1–5)	Faithfulness (1–5)	Creativity (1–5)
Baseline (DistilGPT2)	1.0	1.0	1.0
Fine-Tuned	3.0	2.0	2.0

=> The fine-tuned model generates more coherent and readable stories than the baseline. However, it only partially follows the specified control tokens, leading to moderate faithfulness scores.

Limitations

• Limited adherence to tone and setting constraints
• May partially ignore control tokens
• Narrative complexity limited by model size
• Simplistic storytelling patterns
• No advanced safety filtering
• May produce repetitive text for long outputs
• Not trained for real-world advisory use

Bias and Risks

The model inherits biases from:

• The TinyStories dataset
• The original DistilGPT2 training data

Possible limitations include:

• Simplified moral framing
• Limited diversity in narrative themes
• Generic storytelling structure

=> This model is intended for educational and research use only.

Safety Considerations

• No built-in content moderation
• No toxicity filtering
• Outputs should be reviewed before public use