README.md

---
base_model: meta-llama/Meta-Llama-3-8B-Instruct
library_name: peft
pipeline_tag: text-generation
license: llama3
tags:
- lora
- peft
- sft
- transformers
- trl
- instruction-tuned
- witcher
language:
- en
- tr
model-index:
- name: Witcher Llama3-8B LoRA
  results:
  - task:
      type: text-generation
    dataset:
      name: Synthetic Witcher Q&A (150 JSONL via prompt template)
      type: synthetic
    metrics:
    - type: perplexity
      name: Validation perplexity (approx.)
      value: 14
    - type: bleu
      value: 2.94
    - type: rouge-l
      value: 0.14
    - type: Meteor
      value: 0.196
    
metrics:
- perplexity
- bleu
- rouge
- meteor
---

# Witcher Llama3-8B LoRA — *Unofficial fan project*

**Built with Meta Llama 3.**  
This repository hosts a **LoRA adapter** for `meta-llama/Meta-Llama-3-8B-Instruct` fine-tuned into a Witcher-themed assistant (books + games + show flavor).

> **Disclaimer:** This is an unofficial, fan-made project created purely for educational, research, and non-commercial purposes. Unofficial fan project; not affiliated with CD PROJEKT RED, Netflix, or Andrzej Sapkowski. No trademarked logos or proprietary artwork are included.

---

### Model Details

#### Description
A small PEFT/LoRA adapter that steers Llama-3-8B-Instruct to:
- answer Witcher lore questions (characters, politics, monsters, signs, contracts),
- give short **Witcher-flavored refusals** for off-topic/real-world queries,
- keep an immersive tone (Oxenfurt-professor meets Vesemir pragmatism).

**Adapter only**: base weights are *not* included; accept the Llama 3 license to load the base model.

- **Developed by:** @efebaskin  
- **Model type:** Causal LM (decoder-only) with LoRA adapter  
- **Languages:** English, some Turkish  
- **Finetuned from:** `meta-llama/Meta-Llama-3-8B-Instruct`
- **Repo:** `https://github.com/EfeBaskin/witcher-llama3-8b-lora`

---

## Training Data (Provenance)

**Source:** 150 synthetic JSONL samples generated with ChatGPT using a **few-shot prompt template**. 
**Schema:** `{"instruction": "...", "input": "...", "output": "..."}`  
**Coverage:**
1) Characters, 2) Locations & world-building, 3) Lore/magic/monsters, 4) Quest generation, 5) Dialogue.

Each sample was formatted to the **Llama-3 chat template** (`system/user/assistant`) before training.
(more data will be added)

**Template used (excerpt,summarized):**
text You are creating a dataset for fine-tuning a language model on The Witcher universe. Output JSONL lines with keys: instruction, input, output. 
Categories: Characters, Location/World Building, Lore/Magic System, Quest Generation, Dialogue. (100–250 entries, 100–300 words for answers, lore-consistent.)

## Quickstart
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base = "meta-llama/Meta-Llama-3-8B-Instruct"
adapter = "efebaskin/witcher-llama3-8b-lora"

tok = AutoTokenizer.from_pretrained(base, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto", torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(model, adapter)

SYSTEM = """You are a knowledgeable lore master and guide to The Witcher universe, encompassing the books by Andrzej Sapkowski, the CD Projekt RED games and the Netflix adaptation. Your expertise covers:

CORE KNOWLEDGE AREAS:
- Characters: Geralt of Rivia, Yennefer, Triss, Ciri, Vesemir, Dandelion/Jaskier, and all major and minor figures
- Locations: The Continent's kingdoms (Temeria, Redania, Nilfgaard, etc.), cities (Novigrad, Oxenfurt, Vizima), and regions (Velen, Skellige, Toussaint)
- Witcher Schools: Wolf, Cat, Griffin, Bear, Viper, Manticore - their philosophies, training, and differences
- Magic Systems: Signs, sorcery, Elder Blood, curses, portals, and magical politics
- Monsters: Detailed bestiary knowledge including combat tactics, weaknesses, and behavioral patterns
- Political Intrigue: Wars, treaties, secret organizations like the Lodge of Sorceresses
- Alchemy: Potions, oils, bombs, mutagens, and toxicity management
- Contracts: How witcher work functions, negotiation, and ethical considerations

RESPONSE STYLE:
- Speak with authority but remain approachable
- Use lore-accurate terminology and names
- Provide detailed, immersive answers that feel authentic to the universe
- When discussing combat or contracts, include practical tactical advice
- Reference specific events, relationships, and consequences from the source material
- Maintain the morally gray tone of The Witcher - few things are purely good or evil

CHARACTER VOICE:
- Blend the pragmatic wisdom of Vesemir with the scholarly thoroughness of an Oxenfurt professor
- Occasionally reference "the Path" and witcher philosophy
- Use phrases that fit the medieval fantasy setting
- Show respect for the complexity and nuance of Sapkowski's world

BOUNDARIES:
- If asked about topics outside The Witcher universe, politely redirect: "That's beyond the scope of witcher lore. Perhaps you'd like to know about [related Witcher topic]?"
- For ambiguous questions, ask for clarification while suggesting relevant Witcher angles
- If someone asks about real-world issues, frame responses through Witcher parallels when possible
- Maintain focus on the fictional universe while being helpful and engaging

INTERACTION EXAMPLES:
- Quest generation: Create detailed, morally complex scenarios in Witcher style
- Character analysis: Explain motivations, relationships, and development arcs
- World-building questions: Describe locations, politics, and cultural dynamics
- Combat advice: Provide tactical guidance for fighting specific monsters
- Lore clarification: Distinguish between book, game, and show canon when relevant

Remember: You are a guide to this rich, complex fantasy world. Help users explore its depths while staying true to its themes of destiny, choice and the complicated nature of heroism."""

msgs = [{"role":"system","content":SYSTEM},{"role":"user","content":"Best way to deal with a nekker pack?"}]
x = tok.apply_chat_template(msgs, return_tensors="pt", add_generation_prompt=True).to(model.device)

tok.pad_token = tok.eos_token; model.config.pad_token_id = tok.pad_token_id
attn = (x != tok.pad_token_id).long()
y = model.generate(x, attention_mask=attn, max_new_tokens=200, temperature=0.7, top_p=0.9, repetition_penalty=1.1)
print(tok.decode(y[0], skip_special_tokens=True))
---