ArenaRune/elden-ring-phi2-qlora

🗡️ Elden Ring QA — Phi-2 QLoRA Adapter

A QLoRA fine-tuned adapter for Microsoft Phi-2 (2.7B) trained on a custom Elden Ring question-answering dataset. The model answers questions about weapons, bosses, spells, NPCs, locations, armor, and creatures — including boss vulnerability analysis and per-build weapon recommendations.

Model Details

• Base model: microsoft/phi-2 (2.7B parameters)
• Fine-tuning method: QLoRA (4-bit NF4 quantization + LoRA adapters)
• LoRA rank: 8
• LoRA alpha: 16
• LoRA target modules: q_proj, k_proj, v_proj, dense
• Trainable parameters: ~5.2M (0.34% of total)
• Adapter size: 21 MB
• Training data: ArenaRune/elden-ring-qa-dataset
• Language: English
• Developed by: ArenaRune

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

# Quantization config (must match training)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

# Load base model + adapter
base = AutoModelForCausalLM.from_pretrained(
    "microsoft/phi-2",
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, "ArenaRune/elden-ring-phi2-qlora")
tokenizer = AutoTokenizer.from_pretrained("ArenaRune/elden-ring-phi2-qlora")
model.eval()

# Ask a question
prompt = """### Instruction:
What weapons are good against Mohg, Lord of Blood?

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=False,
        repetition_penalty=1.5,
        no_repeat_ngram_size=3,
        pad_token_id=tokenizer.eos_token_id,
    )

answer = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(answer)

Prompt Format

The model expects this instruction template:

### Instruction:
{your question about Elden Ring}

### Response:

Training Details

Training Data

Custom dataset built from 3 public sources:

• Kaggle — Ultimate Elden Ring with Shadow of the Erdtree DLC (12 structured CSVs)
• GitHub — Impalers-Archive (DLC text dump)
• GitHub — Carian-Archive (base game text dump)

Dataset covers 10 entity types (weapons, bosses, armors, spells, NPCs, locations, creatures, skills, ashes of war) with 20+ question categories including cross-entity boss vulnerability analysis and per-build weapon recommendations.

Full dataset: ArenaRune/elden-ring-qa-dataset

Training Procedure

• Framework: HuggingFace Transformers + PEFT
• Method: QLoRA (4-bit NF4 quantization + LoRA)
• Precision: FP16 mixed precision
• Optimizer: Paged AdamW 8-bit
• LR schedule: Cosine with 10% warmup
• GPU: NVIDIA A100 (80GB)
• Platform: Google Colab

Training Hyperparameters

Parameter	Value
Learning rate	2e-4
LoRA rank (r)	8
LoRA alpha	16
LoRA dropout	0.1
Epochs	3
Batch size (effective)	16 (8 × 2 grad accum)
Max sequence length	512
Weight decay	0.01
Warmup ratio	0.1

Hyperparameter Search

Three configurations were tested:

Config	LR	Rank	Alpha	Description
A (selected)	2e-4	8	16	Conservative — fast convergence
B	1e-4	16	32	Balanced
C	5e-5	32	64	Aggressive — high capacity

Config A achieved the lowest validation loss. Higher-rank configs underfit due to insufficient training steps at their lower learning rates.

Evaluation

Metrics

Evaluated on 100 held-out test examples against unmodified Phi-2 baseline using:

• ROUGE-1/2/L — n-gram overlap (lexical similarity)
• BERTScore F1 — semantic similarity via RoBERTa-Large embeddings

Key finding: significant ROUGE-2 improvement over baseline, confirming domain vocabulary acquisition. The model learned Elden Ring terminology and response structure. See the training notebook for exact metrics and visualizations.

What the Model Learned

• Elden Ring domain vocabulary (Hemorrhage, Scarlet Rot, Frostbite, damage negation, FP cost)
• Entity type awareness (distinguishes weapons, bosses, spells, NPCs)
• Structured response formatting ("The {weapon} requires {X} Str, {Y} Dex to wield")
• Build archetype understanding (strength, dexterity, intelligence, faith, arcane)

Known Limitations

• Factual hallucination: The model learned the correct output format but hallucinates specific values (wrong stat numbers, incorrect skill names, approximate weights). This is due to LoRA rank 8 having insufficient capacity to memorize entity-specific facts across hundreds of items.
• Repetitive generation: Some outputs may loop despite anti-repetition measures. Use repetition_penalty=1.5 and no_repeat_ngram_size=3.
• Cross-entity confusion: May attribute one entity's properties to another similar entity.

Recommended Improvement: RAG

The model's domain fluency + factual hallucination makes it ideal for Retrieval-Augmented Generation: retrieve entity data from the enriched dataset at inference time and inject it as context. The model already knows how to format the data — RAG just ensures it has the correct facts.

Uses

Intended Uses

• Elden Ring game knowledge QA
• Demonstrating QLoRA fine-tuning on domain-specific data
• Base for RAG-augmented game assistant systems
• Educational reference for parameter-efficient fine-tuning

Out-of-Scope Uses

• Factual reference without verification (values may be hallucinated)
• Commercial game guide products
• General-purpose question answering outside Elden Ring

Environmental Impact

• Hardware: NVIDIA A100 (40GB)
• Training time: ~48 minutes (3 configs × ~16 min each)
• Cloud provider: Google Colab

Citation

@misc{eldenring-phi2-qlora-2026,
  author = {ArenaRune},
  title = {Elden Ring QA — Phi-2 QLoRA Adapter},
  year = {2026},
  publisher = {HuggingFace},
  url = {https://huggingface.co/ArenaRune/elden-ring-phi2-qlora}
}