README.md

---
license: other
library_name: peft
language:
- en
tags:
- dungeons-and-dragons
- rpg
- lora
- peft
- text-generation
- dnd
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
pipeline_tag: text-generation
inference: true
widget:
  - text: "You are a Dungeons & Dragons assistant. Create a D&D character with the following details: Race: Half-Elf, Class: Bard, Background: Entertainer."
    example_title: "D&D Character Creation"
  - text: "You are a Dungeons & Dragons assistant. Design a D&D adventure hook set in a dark forest with a mysterious cult."
    example_title: "Adventure Hook"
  - text: "You are a Dungeons & Dragons assistant. Create a magical item for D&D 5e that would be suitable for a level 5 rogue."
    example_title: "Magic Item"
model-index:
- name: DeepSeek-R1-Distill-Qwen-7B D&D LoRA
  results: []
---

# DeepSeek-R1-Distill-Qwen-7B Fine-tuned for Dungeons & Dragons

This model is a fine-tuned version of [deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) specifically trained on Dungeons & Dragons content. The model is designed to excel at creating D&D characters, adventures, and other D&D-related content.

## Model Details

- **Base Model:** [deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B)
- **Fine-tuning Method:** LORA (Parameter-Efficient Fine-Tuning)
- **LoRA Rank:** 8
- **LoRA Alpha:** 16
- **Target Modules:** q_proj, k_proj, v_proj, o_proj
- **Training Date:** 2025-05-11
- **Dataset Size:** 500 examples from a curated D&D dataset

## Usage

This is a LoRA adapter that needs to be combined with the base model to work. Here's how to use it:

### Using the Transformers Library

```python
from huggingface_hub import snapshot_download
import torch
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the base model and tokenizer
base_model_id = "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B"
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto",
    use_auth_token=True  # If you're using a private model
)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

# Load the LoRA adapter
adapter_model_id = "chendren/deepseek-dnd-lora"
model = PeftModel.from_pretrained(
    model, 
    adapter_model_id,
    use_auth_token=True  # If you're using a private model
)

# Test generation
prompt = "Create a D&D character with the following details: Race: Half-Elf, Class: Bard, Background: Entertainer"
inputs = tokenizer(f"You are a Dungeons & Dragons assistant. {prompt}", return_tensors="pt").to(model.device)

outputs = model.generate(
    input_ids=inputs["input_ids"],
    attention_mask=inputs["attention_mask"],
    max_new_tokens=500,
    temperature=0.7,
    top_p=0.9,
    top_k=50,
    repetition_penalty=1.1,
    do_sample=True
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```

### Using the Hugging Face Inference API

You can also use this model directly with the Inference API:

```python
import requests

API_URL = "https://api-inference.huggingface.co/models/chendren/deepseek-dnd-lora"
headers = {"Authorization": "Bearer YOUR_API_TOKEN"}

def query(payload):
    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()

output = query({
    "inputs": "You are a Dungeons & Dragons assistant. Create a D&D character with the following details: Race: Half-Elf, Class: Bard, Background: Entertainer",
    "parameters": {
        "max_new_tokens": 500,
        "temperature": 0.7,
        "top_p": 0.9,
        "top_k": 50,
        "repetition_penalty": 1.1,
        "do_sample": True
    }
})
```

## Example Outputs

When prompted to create a D&D character with specific details, the model will generate a complete character sheet with attributes, skills, background story, and more. For example:

**Prompt:** Create a D&D character with the following details: Race: Half-Elf, Class: Bard, Background: Entertainer

**Output:** [The model will generate a detailed character sheet including attributes, skills, spells, personality traits, and background story for a Half-Elf Bard with the Entertainer background]

## Training 

This model was fine-tuned using the following hyperparameters:

- Learning rate: 5e-5
- Epochs: 1
- Batch size: 1
- Gradient accumulation steps: 4
- Maximum sequence length: 256

## License

This model inherits the license of the base model, [deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B).