README.md

---
license: mit
base_model: meta-llama/Llama-3.2-3B-Instruct
model_name: NovelCrafter-LoRA
tags:
- text-generation
- lora
- peft
- creative-writing
- fine-tuned
- academic
- research
library_name: peft
pipeline_tag: text-generation
language:
- en
---

# NovelCrafter-LoRA: A Fine-Tuned Language Model for Creative Writing

## Model Description

**NovelCrafter-LoRA** is a Low-Rank Adaptation (LoRA) fine-tuned model designed to assist with creative and literary text generation. This adapter is trained on top of Meta's Llama 3.2 Instruct model family and is optimized for generating coherent, stylistically rich narrative text.

### Model Summary

| Property | Value |
|----------|-------|
| **Model Type** | LoRA Adapter (PEFT) |
| **Base Model** | `meta-llama/Llama-3.2-3B-Instruct` (GPU) / `meta-llama/Llama-3.2-1B-Instruct` (CPU) |
| **Language** | English |
| **License** | MIT License |
| **Fine-tuning Method** | LoRA (Low-Rank Adaptation) |
| **Training Steps** | 19 |
| **Last Updated** | 2026-01-20 |

## Technical Specifications

### LoRA Configuration

| Parameter | Value |
|-----------|-------|
| **Rank (r)** | 8 |
| **Alpha** | 32 |
| **Dropout** | 0.05 |
| **Target Modules** | `q_proj`, `v_proj` |
| **Bias** | None |
| **Task Type** | Causal Language Modeling |

### Training Configuration

| Parameter | Value |
|-----------|-------|
| **Batch Size** | 1 (per device) |
| **Gradient Accumulation Steps** | 8 |
| **Learning Rate** | 5e-5 |
| **Warmup Steps** | 100 |
| **Optimizer** | AdamW (torch) |
| **Epochs per Training Unit** | 3 |

## How to Use

### Installation

First, install the required dependencies:

```bash
pip install transformers peft torch accelerate
```

### Loading the Model

#### Option 1: Using PEFT (Recommended)

```python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Choose base model based on your hardware
# GPU (recommended): meta-llama/Llama-3.2-3B-Instruct
# CPU: meta-llama/Llama-3.2-1B-Instruct
BASE_MODEL = "meta-llama/Llama-3.2-3B-Instruct"
ADAPTER_REPO = "a-01a/novelCrafter"

# Load base model
model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float16,  # Use float32 for CPU
    device_map="auto",
)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, ADAPTER_REPO)

# Load tokenizer from adapter repo (includes custom chat template)
tokenizer = AutoTokenizer.from_pretrained(ADAPTER_REPO)

print("Model loaded successfully!")
```

#### Option 2: Merging Adapter into Base Model

```python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE_MODEL = "meta-llama/Llama-3.2-3B-Instruct"
ADAPTER_REPO = "a-01a/novelCrafter"

# Load and merge
base_model = AutoModelForCausalLM.from_pretrained(BASE_MODEL, torch_dtype=torch.float16)
model = PeftModel.from_pretrained(base_model, ADAPTER_REPO)
model = model.merge_and_unload()  # Merge LoRA weights into base model

tokenizer = AutoTokenizer.from_pretrained(ADAPTER_REPO)
```

### Generating Text

```python
def generate_text(prompt, max_new_tokens=512):
    '''Generate text using the fine-tuned model.'''
    messages = [
        {
            "role": "system",
            "content": "You are a skilled creative writing assistant. Write engaging, "
                       "descriptive prose with attention to character development and narrative flow."
        },
        {
            "role": "user",
            "content": prompt
        }
    ]
    
    # Apply chat template
    formatted_prompt = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )
    
    # Tokenize
    inputs = tokenizer(formatted_prompt, return_tensors="pt").to(model.device)
    
    # Generate
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            temperature=0.7,
            top_p=0.9,
            do_sample=True,
            pad_token_id=tokenizer.eos_token_id,
        )
    
    # Decode and return
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return response.split("assistant")[-1].strip()

# Example usage
result = generate_text(
    "Write an opening paragraph for a mystery novel set in Victorian London."
)
print(result)
```

### Using with Transformers Pipeline

```python
from transformers import pipeline

generator = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    device_map="auto",
)

output = generator(
    "Once upon a time in a distant kingdom,",
    max_new_tokens=200,
    temperature=0.7,
)[0]["generated_text"]

print(output)
```

## Intended Use

### Primary Use Cases

- **Creative Writing Assistance**: Generating narrative prose, story continuations, and creative text
- **Academic Research**: Studying fine-tuning techniques for creative text generation
- **Educational Purposes**: Learning about LoRA adapters and PEFT methods

### Out-of-Scope Uses

This model is **NOT** intended for:

- Commercial applications without proper licensing
- Generating misleading or deceptive content
- Any application that violates ethical guidelines
- Production systems without human oversight

## Limitations and Biases

- The model may generate repetitive or incoherent text for very long outputs
- Quality depends heavily on the prompt structure and specificity
- May exhibit biases present in the base model and training data
- Not optimized for factual accuracy or real-world knowledge tasks
- Performance varies based on the base model variant used

## Ethical Considerations

⚠️ **ACADEMIC AND RESEARCH USE ONLY**

This model is released strictly for academic research and educational purposes. By using this model, you agree to:

1. **Non-Commercial Use**: This model may not be used for commercial purposes without explicit written permission.

2. **Responsible Use**: Users must ensure generated content does not cause harm, spread misinformation, or violate any laws.

3. **Attribution**: Any academic publications or research using this model should provide appropriate citation.

4. **No Malicious Use**: The model must not be used to generate harmful, abusive, or illegal content.

5. **Human Oversight**: All generated content should be reviewed by humans before any public distribution.

6. **Compliance**: Users must comply with the base model's (Meta Llama) license terms and acceptable use policy.

## Citation

If you use this model in your research, please cite:

```bibtex
@misc{NovelCrafter-lora-2025,
  title={NovelCrafter-LoRA: A Fine-Tuned Language Model for Creative Writing},
  author={a-01a},
  year={2025},
  publisher={Hugging Face},
  url={https://huggingface.co/a-01a/novelCrafter}
}
```

## Contact

For questions, issues, or collaboration inquiries, please open an issue on the Hugging Face repository.

---

**Disclaimer**: This model is provided "as-is" without warranty of any kind. The authors are not responsible for any misuse or harm caused by the use of this model.