Proteinrequired/email-triage-lora

Email-Triage LoRA (Llama-3.2-3B-Instruct)

A LoRA adapter (r=16) fine-tuned via Behavioral Cloning to triage corporate emails by selecting one of three tools — route_to_human, auto_reply, or ask_for_clarification — returned as a strict JSON tool call.

• Base model: unsloth/Llama-3.2-3B-Instruct
• Method: Behavioral Cloning (Unsloth + Hugging Face TRL), 60 steps
• Adapter: LoRA, r=16
• Project: Enterprise Email Triage Simulator
• Live demo (Gemini-backed UI): https://huggingface.co/spaces/Proteinrequired/enterprise-email-triage-v2

Results — held-out tool-selection accuracy

Evaluated on a 21-email held-out test set (canonical 100-email dataset, intent-stratified 80/20 split, seed 42; BC trained only on the 79 train emails). Metric = did the policy pick the reward system's optimal tool for each email.

Policy	Held-out accuracy (N=21)
Random choice	33.0%
Always route_to_human	47.6%
Rule-based heuristic	76.2% (16/21)
This adapter	71.4% (15/21)

The adapter is perfect on every route_to_human intent but over-escalates routine (auto_reply) and ambiguous (ask_for_clarification) mail — a known Behavioral-Cloning artifact (trained on reward-positive rollouts skewed toward escalation). It lands within one example of a hand-tuned rule baseline and far above the random / always-escalate floors. Reproduce with run_trained_eval.py.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "unsloth/Llama-3.2-3B-Instruct"
adapter = "Proteinrequired/email-triage-lora"

tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)

The model expects the system/user prompt format defined in run_trained_eval.py and responds with a JSON object: {"tool": "<route_to_human|auto_reply|ask_for_clarification>"}.

Limitations

• Trained on synthetic corporate emails; not validated on real inboxes or other domains.
• 3B model — needs a GPU for low-latency inference. The live demo uses Gemini via API for CPU-only hosting; this adapter is the project's open-weights research artifact.
• Optimizes the project's reward policy; not a general-purpose safety/abuse classifier.