pushpam14 commited on
Commit
c5afe7c
Β·
verified Β·
1 Parent(s): 9e88762

Add submission strengths and data mix summary

Browse files
Files changed (1) hide show
  1. README.md +15 -0
README.md CHANGED
@@ -38,6 +38,21 @@ Per `themes.md` Theme #3.1: *"environments that require real interaction with to
38
  - βœ… **Enterprise workflow nuance**: versioning, deprecation, backward compatibility
39
  - βœ… **Verifiable reward**: every step has a deterministic, objective grader
40
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
  ## Architecture: Phase 1 β†’ Phase 2 β†’ Phase 3
42
 
43
  | Phase | What the agent does | Task examples |
 
38
  - βœ… **Enterprise workflow nuance**: versioning, deprecation, backward compatibility
39
  - βœ… **Verifiable reward**: every step has a deterministic, objective grader
40
 
41
+ ## What Makes This Submission Strong
42
+
43
+ This is not a static prompt benchmark. It is a runnable OpenEnv environment with hidden ground truth, stateful episodes, objective rewards, real training, and public proof artifacts.
44
+
45
+ | Area | What is included |
46
+ |---|---|
47
+ | **Environment depth** | 9 tasks across detection, downstream impact tracing, and backward-compatible fix verification |
48
+ | **Episode/data mix** | Seeded synthetic enterprise API scenarios: OpenAPI specs, payloads, version diffs, consumer service graphs, and migration candidates |
49
+ | **Reward richness** | 14 independent reward signals covering correct findings, proximity, duplicates, false positives, missed consumers, malformed patches, broken consumers, and anti-spam |
50
+ | **Training evidence** | 300 GRPO steps on Qwen2.5-7B + LoRA, public WandB run, reward curve, training state JSON, full logs, and trained adapter on Hugging Face |
51
+ | **Before/after evaluation** | Three-way comparison: untrained Qwen2.5-72B, untrained Qwen2.5-7B, and trained Qwen2.5-7B + LoRA |
52
+ | **Headline improvement** | `detect_breaking_changes`: 0.01 β†’ 0.67 for the same 7B base model after GRPO |
53
+
54
+ The environment uses generated, deterministic scenarios rather than a scraped external dataset. That is intentional: every episode has known ground truth, which makes the reward signal auditable and lets judges reproduce the same task with a fixed `seed`.
55
+
56
  ## Architecture: Phase 1 β†’ Phase 2 β†’ Phase 3
57
 
58
  | Phase | What the agent does | Task examples |