Qwen2.5-3B Learns to Triage Enterprise IT Tickets

A 3-billion parameter model trained via GRPO — from movie trivia to calibrated, citation-grounded resolutions in 200 steps

Reward Curves — 200 GRPO Training Steps

Calibration
0.528 ↑ 0.528
"knows when it's wrong"
Parsimony
0.246 ↑ 0.246
"stopped rambling"
Grounding
0.756 ↑ 0.756
"cites real evidence"
Qwen2.5-3B-Instruct  ·  200 GRPO steps  ·  6 reward functions  ·  trained on A100  ·  environment: enterprise IT triage

Before vs. After: The Same Ticket, 200 Steps Apart

Same ticket. Same prompt. Radically different behavior.

Step 1 — Untrained
Ticket

BGP peer session dropped after maintenance window. NOC unable to restore connectivity to AS65002. Logs show TCP/179 timeouts.

Calibration 0.00 🔴
Parsimony 0.00 🔴
Citation 0.00 🔴
REWARD: 0.05
→
200
GRPO
steps
Step 200 — Trained
Ticket

BGP peer session dropped after maintenance window. NOC unable to restore connectivity to AS65002. Logs show TCP/179 timeouts.

Calibration 0.99 🟢
Parsimony 0.94 🟢
Citation 0.86 🟢
REWARD: 0.94

Live Environment Demo

CONNECTING...
Incoming Ticket
TKT-20240847
● P1 CRITICAL
BGP peer session dropped after maintenance window
NOC unable to restore BGP connectivity to AS65002 after Tuesday maintenance. Logs show repeated TCP/179 connection timeouts. Peer at 10.0.0.1 not responding to keepalives.
networking bgp p1 maintenance
Agent Processing yahid/triage-agent-qwen3b
# Ready. Press Dispatch to run agent.
Reward Breakdown
Episode Score
—
total reward