Add model card with honest held-out results

README.md

@@ -0,0 +1,68 @@
+---
+license: llama3.2
+base_model: unsloth/Llama-3.2-3B-Instruct
+library_name: peft
+tags:
+  - lora
+  - peft
+  - email-triage
+  - tool-calling
+  - text-classification
+pipeline_tag: text-generation
+---
+
+# Email-Triage LoRA (Llama-3.2-3B-Instruct)
+
+A LoRA adapter (`r=16`) fine-tuned via Behavioral Cloning to triage corporate emails by
+selecting one of three tools — `route_to_human`, `auto_reply`, or `ask_for_clarification` —
+returned as a strict JSON tool call.
+
+- **Base model:** `unsloth/Llama-3.2-3B-Instruct`
+- **Method:** Behavioral Cloning (Unsloth + Hugging Face TRL), 60 steps
+- **Adapter:** LoRA, `r=16`
+- **Project:** [Enterprise Email Triage Simulator](https://github.com/vaishali-strategy/email-triage)
+- **Live demo (Gemini-backed UI):** https://huggingface.co/spaces/Proteinrequired/enterprise-email-triage-v2
+
+## Results — held-out tool-selection accuracy
+
+Evaluated on a **21-email held-out test set** (canonical 100-email dataset, intent-stratified
+80/20 split, seed 42; BC trained only on the 79 train emails). Metric = did the policy pick the
+reward system's optimal tool for each email.
+
+| Policy | Held-out accuracy (N=21) |
+| :--- | :---: |
+| Random choice | 33.0% |
+| Always `route_to_human` | 47.6% |
+| Rule-based heuristic | **76.2%** (16/21) |
+| **This adapter** | **71.4%** (15/21) |
+
+The adapter is **perfect on every `route_to_human` intent** but **over-escalates** routine
+(`auto_reply`) and ambiguous (`ask_for_clarification`) mail — a known Behavioral-Cloning artifact
+(trained on reward-positive rollouts skewed toward escalation). It lands within one example of a
+hand-tuned rule baseline and far above the random / always-escalate floors. Reproduce with
+[`run_trained_eval.py`](https://github.com/vaishali-strategy/email-triage/blob/v2-rebuild/run_trained_eval.py).
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+base = "unsloth/Llama-3.2-3B-Instruct"
+adapter = "Proteinrequired/email-triage-lora"
+
+tok = AutoTokenizer.from_pretrained(base)
+model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
+model = PeftModel.from_pretrained(model, adapter)
+```
+
+The model expects the system/user prompt format defined in
+[`run_trained_eval.py`](https://github.com/vaishali-strategy/email-triage/blob/v2-rebuild/run_trained_eval.py)
+and responds with a JSON object: `{"tool": "<route_to_human|auto_reply|ask_for_clarification>"}`.
+
+## Limitations
+
+- Trained on synthetic corporate emails; not validated on real inboxes or other domains.
+- 3B model — needs a GPU for low-latency inference. The live demo uses Gemini via API for
+  CPU-only hosting; this adapter is the project's open-weights research artifact.
+- Optimizes the project's reward policy; not a general-purpose safety/abuse classifier.