Upload blog_post.md

blog_post.md

@@ -0,0 +1,64 @@
+# Teaching a 0.5B Model to Be an Executive Assistant
+
+*An OpenEnv Hackathon submission — built in 36 hours, trained in 90 minutes.*
+
+---
+
+## The setup
+
+Every executive's inbox is the same problem on repeat. A meeting request comes in. There's a calendar conflict you have to spot. You write a polite reply, propose a time that actually works, and don't double-book anyone. It's not hard — it's just *fiddly*, and LLMs are surprisingly bad at it because the task fuses three separate skills: structured calendar reasoning, professional tone, and conflict resolution.
+
+I built **ExecAssist**, an OpenEnv environment that simulates exactly that loop, and trained a small model on it with GRPO. The results were dramatic.
+
+## The environment
+
+ExecAssist gives an agent a realistic snapshot of an executive's morning: incoming emails (with sender, subject, body, priority), the existing calendar, working hours, and contact info. The agent has to produce a single JSON action — a written email reply, a calendar action (`book` / `propose_alternatives` / `reschedule` / `decline`), and the meeting details if booking.
+
+There are three tasks of escalating difficulty:
+
+- **Easy** — single email, clear availability. *Just don't mess up the basics.*
+- **Medium** — the requested time conflicts with an existing meeting. *Spot it, propose alternatives.*
+- **Hard** — three emails, multi-party coordination, priority conflicts. *Actually plan.*
+
+Reward is a weighted blend of three independent graders (email quality, scheduling correctness, conflict resolution) plus four anti-reward-hacking penalties — short emails, missing meeting details, generic templated replies, and overly long responses all get docked. Multiple independent reward functions make the signal hard to game, which matters because GRPO will absolutely find any shortcut you leave open.
+
+## The training run
+
+I trained `Qwen2.5-0.5B-Instruct` — a tiny 500M-parameter model — using TRL's `GRPOTrainer` for 3 epochs over 90 collected scenarios on a free Colab T4. Total training time: **about 90 minutes.**
+
+Two specific config choices mattered a lot. The first run (1 epoch, lr=5e-6, no KL term) **collapsed** — the model found a single safe response that scored 0.2 every time and refused to explore further. Classic GRPO failure mode. Adding `beta=0.1` (KL penalty against the base model), dropping the learning rate to `1e-6`, and bumping `num_generations` to 8 fixed it cleanly.
+
+## The result
+
+Across 10 evaluation samples per task:
+
+| Task   | Baseline (untrained 0.5B) | Trained (GRPO) | Improvement |
+|--------|---------------------------|----------------|-------------|
+| Easy   | 0.345                     | **0.995**      | **+188%**   |
+| Medium | 0.227                     | **0.745**      | **+228%**   |
+| Hard   | 0.249                     | **0.737**      | **+196%**   |
+
+The headline isn't just the means — it's the **variance collapse**. Baseline scores on the easy task ranged from 0.0 to 0.65 (the model rolled the dice). Trained scores: nine out of ten samples hit exactly 1.0. That's the model learning *how the task works*, not just getting lucky.
+
+The training curve tells the same story. Reward starts oscillating around 0.1–0.4 in the first 50 steps, climbs steadily through the middle, and stabilizes between 0.6 and 0.9 in the final third. First quartile mean: 0.390. Last quartile mean: **0.648**. A 66% lift *during* training, on top of the much larger gap between the trained model and the untrained one at evaluation time.
+
+![Training results](./training_results.png)
+
+## Why this is interesting beyond the numbers
+
+There's a nice secondary result hiding in here. As a sanity check, I'd previously run the same three tasks against a frontier free-tier model — Nemotron 120B via OpenRouter, with no task-specific training — using the original LLM-judge reward path. It scored an average of **0.337** across the three tasks. After 90 minutes of GRPO, a model **240× smaller** is hitting **0.83 average** on the same environment.
+
+That's the point of training-environment design. A well-shaped reward signal lets a tiny model beat a frontier model on a specific task, in under two hours, on free hardware.
+
+## Try it yourself
+
+- **Live environment:** [`devanshudon-exec-assist.hf.space/docs`](https://devanshudon-exec-assist.hf.space/docs) — interact with the API directly via Swagger
+- **Tasks:** `POST /reset?task=easy|medium|hard` then `POST /step` with your action
+- **Baseline:** `python inference.py` reproduces the untrained scores
+- **Training:** the Colab notebook is in the repo — set runtime to T4, run all, ~50 minutes
+
+The environment is genuinely hard in interesting ways. Feel free to break it.
+
+---
+
+*Built for the OpenEnv Hackathon (Apr 2026), Theme #3.2 — Personalized Tasks.*