Spaces:
Sleeping
Sleeping
Add submission strengths and data mix summary
Browse files
BLOG.md
CHANGED
|
@@ -46,6 +46,20 @@ GitHub README: https://github.com/kumarpushpam17-personal/Hackathon/blob/main/ap
|
|
| 46 |
|
| 47 |
The end result is a training environment where an LLM agent interacts with a simulated enterprise API ecosystem and learns to reason about contract changes, downstream consumers, and backward-compatible fixes.
|
| 48 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
## The story in 30 seconds
|
| 50 |
|
| 51 |
It is Friday evening. A backend engineer makes what looks like a small API cleanup:
|
|
|
|
| 46 |
|
| 47 |
The end result is a training environment where an LLM agent interacts with a simulated enterprise API ecosystem and learns to reason about contract changes, downstream consumers, and backward-compatible fixes.
|
| 48 |
|
| 49 |
+
## What makes this submission strong
|
| 50 |
+
|
| 51 |
+
This is not a static prompt benchmark. It is a runnable OpenEnv environment with hidden ground truth, stateful episodes, objective rewards, real training, and public proof artifacts.
|
| 52 |
+
|
| 53 |
+
| Area | What is included |
|
| 54 |
+
|---|---|
|
| 55 |
+
| Environment depth | 9 tasks across detection, downstream impact tracing, and backward-compatible fix verification |
|
| 56 |
+
| Episode/data mix | Seeded synthetic enterprise API scenarios: OpenAPI specs, payloads, version diffs, consumer service graphs, and migration candidates |
|
| 57 |
+
| Reward richness | 14 independent reward signals covering correct findings, proximity, duplicates, false positives, missed consumers, malformed patches, broken consumers, and anti-spam |
|
| 58 |
+
| Training evidence | 300 GRPO steps on Qwen2.5-7B + LoRA, public WandB run, reward curve, training state JSON, full logs, and trained adapter on Hugging Face |
|
| 59 |
+
| Before/after evaluation | Three-way comparison: untrained Qwen2.5-72B, untrained Qwen2.5-7B, and trained Qwen2.5-7B + LoRA |
|
| 60 |
+
|
| 61 |
+
The environment uses generated, deterministic scenarios rather than a scraped external dataset. That is intentional: every episode has known ground truth, which makes the reward signal auditable and lets judges reproduce the same task with a fixed `seed`.
|
| 62 |
+
|
| 63 |
## The story in 30 seconds
|
| 64 |
|
| 65 |
It is Friday evening. A backend engineer makes what looks like a small API cleanup:
|