How a Self-Taught Builder with No Coding Background Made It to the Meta PyTorch Grand Finale
By Hariharan | Meta × HuggingFace × PyTorch OpenEnv Hackathon Grand Finale | April 25-26, 2026
Two months ago I didn't know what an RL environment was.
I started vibe-coding with Claude in February. No CS degree, no formal ML background — just curiosity and a lot of questions.
When I heard about this hackathon, I didn't ask "what should I build to impress the judges." I asked Claude a different question:
"Why do these companies run hackathons? What problem keeps their engineers up at night?"
The answer: the hardest unsolved problem in agentic AI isn't capability — it's reliability. Agents that confidently follow stale instructions. Agents that ignore mid-task context shifts. Agents that were told "switch to image recognition" and kept preparing the text dataset anyway.
I looked around and saw every major infra company — building agentic memory, agentic payments, agentic workflows — quietly fighting this same problem. That external validation told me the idea was real.
So I built DriftEnv.
What is DriftEnv?
An RL environment that puts AI agents directly in that situation — vague instructions, mid-task shifts — and trains the failure behavior out.
- 25 scenarios across 5 ML workflow domains
- Each scenario: vague initial instruction + mid-task context shift
- Agent must interpret correctly, then pivot when things change
- 3 difficulty tiers: easy (1 step), medium (2 steps), hard (3 steps)
- OpenEnv compliant, deployed on HF Spaces
Links:
- Live Space: https://huggingface.co/spaces/harims95/driftenv
- Trained Model: https://huggingface.co/harims95/driftenv-qwen1.5b-sft-grpo
- GitHub: https://github.com/harims95/driftenv
What happened during training
I'm going to be honest — nothing worked the first time.
We ran 5 training experiments over 2 days:
Run 1: Basic GRPO, 150 steps. Reward stuck at 0.245. Discovered R_pivot (40% of reward) was 0.0 the entire time. The model never saw a context shift during training — we only trained on easy (1-step) tasks. No pivot = no learning on the most important signal.
Run 2: Shorter completions (80 tokens), length bonus. Holdout score improved to 0.271. +23% over untrained. But model still generated maximum length every time.
Runs 3 and 4: Added length penalties. -0.05, then -0.30. clipped_ratio stayed 1.0 throughout.
The discovery: RL penalty alone cannot break pre-trained verbosity. The model learned to be chatty during pre-training on trillions of tokens. 50 steps of GRPO can't override that. This matches Guide Section 3 exactly — "RL often needs some warm start."
The fix: SFT warm-start. 130 steps of supervised fine-tuning on concise one-sentence examples FIRST. Then GRPO on top.
Before SFT: model generated 52-word essays every time. After SFT: model generated 19-word focused responses.
Run 5 (SFT + GRPO): Holdout score jumped to 0.343. +55.9% over untrained. One scenario beat the 72B reference model.
The results
| Model | Holdout Score | Improvement |
|---|---|---|
| Qwen 1.5B Untrained | 0.220 | baseline |
| GRPO only | 0.271 | +23% |
| SFT + GRPO | 0.343 | +55.9% |
| Qwen 72B Reference | 0.436 | ceiling |
Our trained 1.5B reached 78.6% of the 72B model's performance on unseen holdout scenarios. One scenario (deployment domain) scored 0.514 — matching the 72B reference with a model 50x smaller.
What I learned — not just about RL
1. SFT before RL is not optional for behavior change. We proved this empirically across 4 ablations. The guide says it in Section 3. We had to learn it the hard way.
2. Log reward components separately. R_pivot was 0.0 for 4 entire runs. Invisible in the total reward number. Only component-level logging revealed it.
3. Small models + many iterations beats one big run. 5 training runs on Qwen 1.5B taught us more than one run on a 70B model ever could. Each run revealed something the previous one didn't.
4. Keyword rewards are exploitable. Agents learn to echo instruction words to score well. Anti-hack filtering fixed it. Impact was measurable.
The bigger lesson
I'm going to be brutally honest: I don't know a single line of code. The idea is mine. Claude helped me build it.
Two months ago if someone told me I'd be sitting at the Meta × HuggingFace Grand Finale next to IIT, BITS Pilani and JP Morgan engineers — I would have laughed.
Today I'm here. With a fever and a cold, having run 5 training experiments, having debugged broken reward functions at midnight, having fought through Kaggle dependency hell for 3 hours before switching to Colab.
The gap between "has an idea" and "ships something real" is smaller than it's ever been. You don't need to know how to code. You need a problem worth solving and the hunger to figure things out.
If I can do it, anyone can.
Built at Meta × HuggingFace × PyTorch OpenEnv Hackathon Grand Finale April 25-26, 2026 | Scaler School of Technology, Bangalore Grand Finale qualifier from 52,000+ teams.