Educational Content Rewriter — LLaMA 3.2 3B LoRA

Model Summary

This is a LoRA fine-tuned version of Meta's LLaMA 3.2 3B base model, trained to rewrite confusing educational content into clearer, more accessible alternatives. The model supports six targeted rewrite modes, each designed to address a specific clarity need in educational writing.

This model was developed as part of a structured NLP/LLM learning journey, serving as the core engine for the Teaching Quality Analyzer capstone project.

Model Details

Property	Value
Base model	meta-llama/Llama-3.2-3B
Model type	Causal Language Model (Decoder-only)
Fine-tuning method	QLoRA (4-bit quantisation + LoRA)
LoRA rank	16
LoRA alpha	32
Target modules	q_proj, v_proj, k_proj, o_proj
Trainable parameters	3,407,872 (0.45% of total)
Training precision	BFloat16
License	Apache 2.0

Intended Use

Primary Use Cases

Rewriting dense, jargon-heavy educational text into clearer alternatives
Helping educators improve the accessibility of their content before publishing or recording
Assisting content creators in adapting technical material for different audience levels
Powering educational content quality tools and dashboards

Out-of-Scope Use Cases

General-purpose text generation
Creative writing or fiction
Medical, legal, or financial advice
Any application requiring factual accuracy guarantees (the model may hallucinate or alter facts)

Rewrite Modes

The model supports six distinct rewrite modes, specified via the system prompt at inference time.

Mode	Description
Default	General clarity improvement — improves readability and flow while preserving meaning
Simpler	Accessibility rewrite — replaces jargon with everyday language, targets non-expert readers
Add Example	Adds one concrete, domain-relevant example to illustrate the concept
Concise	Reduces word count while preserving all key information
Step by Step	Breaks the explanation into clearly numbered steps
Add Analogy	Adds a real-world comparison that makes abstract concepts concrete

Training Data

The model was fine-tuned on a custom synthetic dataset of 846 educational rewrite pairs generated using the Anthropic Claude API, following the methodology established in the Alpaca data generation paper (Taori et al., 2023).

Property	Value
Total examples	846
Source passages	141 (66 Wikipedia + 75 arXiv abstracts)
Modes per passage	6
Domain scope	All domains (computer science, biology, physics, mathematics, economics, chemistry, medicine)
Passage length	Mixed — sentence (20%), paragraph (60%), section (20%)
Rewrite generation	Claude API (claude-sonnet-4-5) with manual seed examples
Prompt style	Dynamic — 49% short system prompts, 51% detailed system prompts
Train split	612 examples (70%)
Validation split	138 examples (17%)
Test split	96 examples (13%)
Split strategy	Stratified at passage level (all 6 modes for a passage go to the same split)

Data Generation Methodology

Source passages were collected from Wikipedia technical articles and arXiv paper abstracts across diverse academic domains. Each passage was then fed to the Claude API with mode-specific prompts to generate six targeted rewrites. A small set of manually written seed examples was included to anchor the generation quality. This approach mirrors the synthetic data generation methodology used in Stanford Alpaca and Microsoft Orca.

Training Configuration

Hyperparameter	Value
Epochs	3
Learning rate	2e-4
LR scheduler	Cosine
Warmup steps	50
Batch size	2
Gradient accumulation steps	8
Effective batch size	16
Weight decay	0.01
Max gradient norm	1.0
Max sequence length	512
Quantisation	4-bit NF4 (QLoRA)
Training framework	TRL SFTTrainer
Hardware	NVIDIA Tesla T4 (16GB)
Training time	150 minutes

Evaluation Results

Training Curves

Epoch	Training Loss	Validation Loss
1	1.737	1.183
2	1.149	1.086
3	1.044	1.065

The gap between training and validation loss at epoch 3 is 0.021, indicating minimal overfitting despite the relatively small dataset size.

Mode-Specific Qualitative Evaluation

Qualitative evaluation was conducted on a held-out test passage across all six modes.

Mode	Assessment	Notes
Default	Good	Clear structure, accessible language, appropriate length
Simpler	Partial	Reduces complexity but occasionally retains technical vocabulary
Add Example	Partial	Adds examples but domain relevance is inconsistent
Concise	Good	Consistently produces shorter outputs (49% reduction observed)
Step by Step	Good	Produces structured numbered steps following the source mechanism
Add Analogy	Good	Generates relevant real-world comparisons with clear mappings

Known Limitations

Simpler mode occasionally retains jargon from the source text. Root cause: training data generated by a large language model (Claude) that itself uses technical vocabulary. Recommended fix: regenerate Simpler mode training pairs with stricter jargon constraints.
Add Example mode sometimes adds examples from unrelated domains. Root cause: insufficient domain-specificity constraints in the generation prompts. Recommended fix: regenerate Add Example pairs with explicit domain-matching requirements.
Model size: At 3B parameters, the model shows strong instruction-following on most modes but may struggle with highly complex or ambiguous inputs. Upgrading to a 7B model is expected to improve consistency across all modes.
Factual accuracy: The model may alter facts or introduce inaccuracies in rewrites. All outputs should be reviewed before publication.
Language: English only. Not tested on multilingual inputs.

How to Use

Loading the Model

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

model_name = "meta-llama/Llama-3.2-3B"
adapter_name = "ray-2908/educational-rewriter-lora"

# 4-bit quantisation config
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

# Load base model
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token

base_model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    device_map="auto",
)

# Load LoRA adapters
model = PeftModel.from_pretrained(base_model, adapter_name)
model.eval()

Inference

def generate_rewrite(model, tokenizer, text, mode, max_new_tokens=256):
    
    mode_instructions = {
        "Default": """You are an expert educational content rewriter.
Rewrite the text to be clearer and easier to understand.
Improve readability while preserving the original meaning exactly.
Output ONLY the rewrite, nothing else.""",

        "Simpler": """You are an expert educational content rewriter.
Rewrite the text so a 16-year-old with no technical background can understand it.
Replace ALL technical terms with everyday words.
Use short sentences. Maximum 15 words per sentence.
Never use acronyms or jargon.
If a technical term is unavoidable, explain it in brackets.
Output ONLY the rewrite, nothing else.""",

        "Add Example": """You are an expert educational content rewriter.
Rewrite the text and add ONE concrete example that directly illustrates the concept.
The example MUST be about the exact same topic as the input text.
Do not use unrelated examples from different domains.
Format: original explanation + "For example, ..."
Output ONLY the rewrite with example, nothing else.""",

        "Concise": """You are an expert educational content rewriter.
Rewrite the text using fewer words while keeping the same meaning.
Remove redundancy and filler phrases.
Do not remove important information.
Output ONLY the shorter rewrite, nothing else.""",

        "Step by Step": """You are an expert educational content rewriter.
Break down HOW THIS CONCEPT WORKS into clear numbered steps.
Each step must explain part of the mechanism described in the original text.
Do NOT explain history. Do NOT add information not in the original text.
Format strictly as:
1. [first part of the mechanism]
2. [second part]
3. [third part]
Output ONLY the numbered steps, nothing else.""",

        "Add Analogy": """You are an expert educational content rewriter.
Rewrite the text and add a real-world analogy that makes the concept concrete.
The analogy must map clearly to the concept in the original text.
Explain why the analogy applies.
Output ONLY the rewrite with analogy, nothing else.""",
    }
    
    system = mode_instructions.get(mode, mode_instructions["Default"])
    prompt = f"<|system|>\n{system}\n<|user|>\nText: {text}\n<|assistant|>\n"
    
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            temperature=0.7,
            top_p=0.95,
            top_k=50,
            do_sample=True,
            repetition_penalty=1.3,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id,
        )
    
    new_tokens = outputs[0][inputs['input_ids'].shape[1]:]
    return tokenizer.decode(new_tokens, skip_special_tokens=True).strip()


# Example usage
text = """Backpropagation computes the gradient of the loss function 
with respect to the weights of the network by applying the chain rule 
of calculus iteratively through each layer."""

for mode in ["Default", "Simpler", "Concise", "Step by Step", "Add Analogy"]:
    rewrite = generate_rewrite(model, tokenizer, text, mode)
    print(f"\n[{mode}]\n{rewrite}")

Prompt Format

The model uses the LLaMA chat template format:

<|system|>
{mode-specific system prompt}
<|user|>
Text: {input text}
<|assistant|>
{rewrite output}

Each mode has a dedicated system prompt that specifies the rewriting objective, constraints, and output format. Using the correct system prompt is essential for good results.

Future Work

The following improvements are planned for subsequent iterations:

Regenerate Simpler and Add Example training pairs with stricter domain-specificity and jargon constraints, then retrain
Upgrade base model to 7B for improved instruction following across all modes
Expand training dataset to 2,000+ examples for better generalisation
Add audience suitability scoring — a formula-based 1-10 score indicating the accessibility level of any rewrite
Mode-specific LoRA adapters — separate adapter per mode for higher per-mode quality at the cost of additional training time

Citation

If you use this model in your research or projects, please cite:

@misc{ray2026rewriter,
  author    = {Anisha Ray},
  title     = {Educational Content Rewriter — LLaMA 3.2 3B LoRA},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/ray-2908/educational-rewriter-lora}
}