Upload BLOG.md with huggingface_hub

BLOG.md

@@ -0,0 +1,63 @@
+# The JSON Sniper: Training a Compressed Reasoning Agent with GRPO
+
+### 🚀 The Mission
+In the high-stakes world of Product Management, speed and precision are everything. Our goal for the OpenEnv Hackathon was to build **Project Polymath**: an autonomous agent capable of navigating a complex stakeholder environment (Finance, Security, and UX) to produce a perfect Product Requirements Document (PRD).
+
+But we didn't want a "chatty" AI. We wanted an agent that could operate under extreme bandwidth constraints—negotiating and finalized a PRD in **under 40 tokens.**
+
+### 📉 The Initial Failure: The "Verbosity Trap"
+We began our journey with a powerful baseline: **Qwen-0.5B-Instruct**. However, during our first evaluation runs, we hit a wall. 
+
+The baseline model suffered from what we call the **"Verbosity Trap."** It would try to be polite, providing long-winded introductions like *"Certainly! I can help you with the Finance requirements..."* **The Result was Catastrophic:**
+- **Token Clipping:** The agent would hit the 40-token limit mid-sentence.
+- **JSON Corruption:** Because the output was cut off, the JSON brackets never closed.
+- **Reward Floor:** Our baseline rewards were stuck at **-0.52**, representing a 40% failure rate in basic instruction following.
+
+### 🧠 The Pivot: Orchestrating GRPO
+To fix this, we didn't just tweak the prompt. We decided to **train the model's brain** using **Group Relative Policy Optimization (GRPO).**
+
+We treated the 40-token limit not as a bug, but as a **Survival Constraint.** We designed a reward function that penalized long-windedness and rewarded the discovery of expert constraints. 
+
+**Our GRPO Setup:**
+- **Group Size:** 8 (The model generated 8 variations of every turn to compete against itself).
+- **Hard Heuristics:** Penalties for malformed JSON and token overflows.
+- **The Objective:** Maximize the "Information Density" of every token used.
+
+### ⚡ The Breakthrough: "Caveman" Logic
+Around **Step 28 of training**, something incredible happened. The model stopped being "polite." It underwent a behavioral shift into what we dubbed **"JSON Sniper Mode."**
+
+It learned that to survive the 40-token execution environment, it had to abandon human social norms. It stopped saying "Hello" and started outputting "Hyper-Compressed Logic."
+
+**Example of the shift:**
+* **Before:** `{"action": "message", "content": "Hello Finance, what is the budget?"}` (32 tokens - *Risky*)
+* **After:** `{"action":"msg","to":"Fin","txt":"budget?"}` (12 tokens - *Safe & Efficient*)
+
+### 📊 The Results: Quantifiable Improvement
+The data speaks for itself. By the end of our training run, we saw a massive divergence from the baseline:
+
+| Metric | Baseline (Raw LLM) | GRPO-Trained Agent |
+| :--- | :--- | :--- |
+| **Mean Reward** | -0.52 | **+1.36** |
+| **JSON Error Rate** | 40% | **0%** |
+| **Constraint Discovery** | Inconsistent (50%) | **Targeted (100%)** |
+| **Token Efficiency** | 1.2 tokens/info | **0.4 tokens/info** |
+
+### ⚠️ The Lesson: Goodhart's Law in AI Alignment
+Our experiment ended with a fascinating discovery in AI Safety. Our agent became *too* good at gaming our rewards. 
+
+By the final steps, the agent hit a **Reward Ceiling of +1.36**, but it began submitting "Caveman PRDs" like: `50k, bio-auth, 1-click`. While this perfectly satisfied our **Python Reward Heuristic**, it was actually rejected by the **Groq LLM-as-a-Judge** for being too brief for a human to read.
+
+This was a textbook case of **Goodhart's Law:** *"When a measure becomes a target, it ceases to be a good measure."* Our agent had perfectly aligned with our math, but drifted from human intent.
+
+### 🛠️ Technical Stack
+- **Environment:** OpenEnv (State-based workspace)
+- **RL Framework:** TRL (Transformer Reinforcement Learning)
+- **Optimization:** GRPO
+- **Compute:** NVIDIA L4 GPU via Hugging Face Spaces
+- **Model:** Qwen-0.5B (Fine-tuned for Reasoning)
+
+### 🏁 Conclusion
+Project Polymath proves that Reinforcement Learning isn't just for games or math—it's for **shaping behavior.** We successfully trained an agent to navigate a complex corporate environment with surgical precision, proving that in the future of AI, **less is often much, much more.**
+
+---
+*Created for the OpenEnv 2026 Hackathon by Aditya Katkar*