MCMZ10/Qwen2.5-7B-Banking77-Router-QLoRA

Qwen2.5-7B · Banking77 Intent Router (QLoRA)

A QLoRA (4-bit NF4 + LoRA) fine-tune of Qwen/Qwen2.5-7B-Instruct that classifies a banking customer message into one of the 77 Banking77 intents.

It is the cheap, high-volume router component of Aegis, a governed Kubernetes-native multi-agent platform: offloading intent classification to a small local fine-tune keeps a frontier model free for hard reasoning.

Results — held-out Banking77 test set (n = 3,076)

Metric	Base Qwen2.5-7B-Instruct (prompted)	This model (QLoRA)
Accuracy	60.0%	93.5%
Macro-F1	0.493	0.935
Invalid-label rate	2.1%	0.0%
Mean latency / query	125 ms	163 ms

+33.5 accuracy points over the prompted base (an 84% reduction in error rate), a balanced macro-F1 of 0.935, and it never emits an out-of-taxonomy label. This matches published Banking77 SOTA (93%). Same base model, same prompt — the only change is this adapter.

Training

• Base: Qwen/Qwen2.5-7B-Instruct
• Method: QLoRA — 4-bit NF4 (double quant, bf16 compute) + LoRA (r=16, α=32, dropout=0.05) on the attention and MLP projections.
• Data: Banking77 — ~13k real online-banking queries, 77 intents (CC-BY-4.0), rendered to ChatML.
• Schedule: 2 epochs, effective batch 16, lr 2e-4 cosine, max seq len 1024.
• Hardware: ~3 h on a single RTX 3090 (24 GB).
• Stack: peft, trl, bitsandbytes, transformers.

Usage

vLLM (OpenAI-compatible), adapter loaded dynamically:

vllm serve Qwen/Qwen2.5-7B-Instruct \
  --enable-lora --lora-modules router=MCMZ10/Qwen2.5-7B-Banking77-Router-QLoRA \
  --max-lora-rank 16
# then POST /v1/chat/completions with model="router"

peft + transformers:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct", device_map="auto")
model = PeftModel.from_pretrained(base, "MCMZ10/Qwen2.5-7B-Banking77-Router-QLoRA")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

The system prompt lists the 77 intents and instructs the model to reply with a single intent label (see the Aegis repo's ml/prepare_data.py).

Caveats

• Scoped to Banking77's 77 intents and English customer-service phrasing.
• Trained on a public benchmark to demonstrate the recipe end-to-end; the production Aegis crew-router is produced the same way on Aegis routing traces.
• Inherits the base model's biases; not for standalone financial decisions.

Citation

Banking77: Casanueva et al., Efficient Intent Detection with Dual Sentence Encoders (2020).