Spaces:
Sleeping
Sleeping
Add separate HF mini-blog markdown and link it from README
Browse files
README.md
CHANGED
|
@@ -258,45 +258,51 @@ python inference.py
|
|
| 258 |
openenv validate
|
| 259 |
```
|
| 260 |
|
| 261 |
-
## Training Results β
|
| 262 |
|
| 263 |
> **Training**: GRPO via TRL + Unsloth Β· **Hardware**: HuggingFace Jobs L4 (24 GB) Β· **Steps**: 300 Β· **Wall-time**: 1 h 56 min
|
| 264 |
-
> **
|
| 265 |
|
| 266 |
### Reward Curve (training progress)
|
| 267 |
|
| 268 |

|
| 269 |
|
| 270 |
-
*Mean episode reward across 300 GRPO training steps on Qwen2.5-7B-Instruct (4-bit + LoRA r=16).
|
| 271 |
|
| 272 |
-
###
|
| 273 |
|
| 274 |

|
| 275 |
|
| 276 |
-
*
|
| 277 |
|
| 278 |
-
| Task | Phase |
|
| 279 |
-
|---|---|---|---|---|
|
| 280 |
-
| `find_type_mismatches` | 1 | 0.75 | 0.75 |
|
| 281 |
-
| `validate_nested_objects` | 1 | 0.99 | 0.57 |
|
| 282 |
-
| **`detect_breaking_changes`** | 1 | **0.01** | **0.
|
| 283 |
-
| `validate_response_schema` | 1 | 0.99 | 0.
|
| 284 |
-
| `validate_cross_field_constraints` | 1 | 0.
|
| 285 |
-
| `validate_auth_request` | 1 | 0.99 | 0.33 |
|
| 286 |
-
| `trace_downstream_blast_radius` | 2 | 0.67 | 0.99 |
|
| 287 |
-
| `propose_backward_compat_fix` | 3 | 0.99 | 0.99 | = |
|
| 288 |
-
| `multi_service_cascade_fix` | 2+3 | 0.99 | 0.99 | = |
|
| 289 |
-
| **Mean** | |
|
| 290 |
|
| 291 |
-
|
| 292 |
|
| 293 |
### What the comparison shows
|
| 294 |
|
| 295 |
-
The
|
| 296 |
|
| 297 |
-
|
| 298 |
-
|
| 299 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 300 |
|
| 301 |
| Phase | WandB Run | Notebook |
|
| 302 |
|---|---|---|
|
|
@@ -326,10 +332,12 @@ This is a genuinely underexplored domain in RL/LLM training β no prior benchma
|
|
| 326 |
| Health check | https://pushpam14-api-contract-validator.hf.space/health |
|
| 327 |
| Trained LoRA adapter | https://huggingface.co/pushpam14/api-contract-validator-grpo-7b |
|
| 328 |
| WandB training run (300 steps) | https://wandb.ai/pushpamsubscriptions-inn/openenv-contract-guardian/runs/gch0eg3k |
|
|
|
|
| 329 |
| Training Notebook (Colab) | [`training/grpo_colab.ipynb`](training/grpo_colab.ipynb) |
|
| 330 |
| Story + Technical Guide | [`ENTERPRISE_CONTRACT_GUARDIAN_STORY.md`](ENTERPRISE_CONTRACT_GUARDIAN_STORY.md) |
|
| 331 |
| GitHub repo | https://github.com/kumarpushpam17-personal/Hackathon |
|
| 332 |
-
|
|
|
|
|
| 333 |
|
| 334 |
Quick test:
|
| 335 |
|
|
|
|
| 258 |
openenv validate
|
| 259 |
```
|
| 260 |
|
| 261 |
+
## Training Results β Three-way comparison (before vs after)
|
| 262 |
|
| 263 |
> **Training**: GRPO via TRL + Unsloth Β· **Hardware**: HuggingFace Jobs L4 (24 GB) Β· **Steps**: 300 Β· **Wall-time**: 1 h 56 min
|
| 264 |
+
> **Inference**: Same temperature (0.7) for all three columns β the comparison is sampling-fair.
|
| 265 |
|
| 266 |
### Reward Curve (training progress)
|
| 267 |
|
| 268 |

|
| 269 |
|
| 270 |
+
*Mean episode reward across 300 GRPO training steps on Qwen2.5-7B-Instruct (4-bit + LoRA r=16). The curve plateaus around 1.0β1.5 because the base 7B model already produces structurally valid actions; the per-task score table below is where the *kind* of improvement becomes visible.*
|
| 271 |
|
| 272 |
+
### Apples-to-apples per-task comparison
|
| 273 |
|
| 274 |

|
| 275 |
|
| 276 |
+
*Three-bar chart: Qwen2.5-72B baseline (dark grey), Qwen2.5-7B baseline (light grey, **same base as trained**), Qwen2.5-7B + LoRA after GRPO (green). The headroom task `detect_breaking_changes` is the standout β both untrained models score 0.01; the trained adapter scores 0.67.*
|
| 277 |
|
| 278 |
+
| Task | Phase | 72B baseline | **7B baseline** | **7B + LoRA (trained)** | Ξ vs 7B base |
|
| 279 |
+
|---|---|---|---|---|---|
|
| 280 |
+
| `find_type_mismatches` | 1 | 0.75 | 0.75 | 0.75 | = |
|
| 281 |
+
| `validate_nested_objects` | 1 | 0.99 | 0.57 | 0.57 | = |
|
| 282 |
+
| **`detect_breaking_changes`** | 1 | **0.01** | **0.01** | **0.67** | **+0.66** π― |
|
| 283 |
+
| `validate_response_schema` | 1 | 0.99 | 0.70 | 0.30 | -0.40 |
|
| 284 |
+
| `validate_cross_field_constraints` | 1 | 0.99 | 0.43 | 0.29 | -0.14 |
|
| 285 |
+
| `validate_auth_request` | 1 | 0.99 | 0.83 | 0.33 | -0.50 |
|
| 286 |
+
| `trace_downstream_blast_radius` | 2 | 0.67 | 0.99 | 0.99 | = |
|
| 287 |
+
| `propose_backward_compat_fix` | 3 | 0.99 | 0.99 | 0.99 | = |
|
| 288 |
+
| `multi_service_cascade_fix` | 2+3 | 0.99 | 0.99 | 0.99 | = |
|
| 289 |
+
| **Mean** | | 0.82 | 0.70 | 0.65 | -0.05 |
|
| 290 |
|
| 291 |
+
Score files: [`../baseline_72b_v2_scores.json`](../baseline_72b_v2_scores.json) (72B), [`../baseline_7b_scores.json`](../baseline_7b_scores.json) (untrained 7B β apples-to-apples baseline), [`../trained_scores.json`](../trained_scores.json) (7B + LoRA after GRPO).
|
| 292 |
|
| 293 |
### What the comparison shows
|
| 294 |
|
| 295 |
+
The middle column (untrained Qwen-7B) is the fair baseline β same base model as the trained one, no adapter. **Any difference between columns 2 and 3 is purely the GRPO training effect.**
|
| 296 |
|
| 297 |
+
**The headline win**:
|
| 298 |
+
|
| 299 |
+
> **`detect_breaking_changes` went from 0.01 β 0.67.** Both untrained models β *including the 10Γ larger 72B* β scored 0.01 on this task. Both models knew where the breaking changes were (they earned the +0.3 proximity reward repeatedly) but **neither could predict `violation_type='breaking_change'` correctly**. After 300 GRPO steps targeting our environment's reward signal, the 7B+LoRA adapter solves the classification 6 of 9 times. **This is RL training value, not model size.**
|
| 300 |
+
|
| 301 |
+
**The trade-off (we're being honest here)**:
|
| 302 |
+
|
| 303 |
+
Three Phase 1 tasks regressed (`validate_response_schema`, `validate_cross_field_constraints`, `validate_auth_request`). Reading the per-step reward trajectories shows the cause: the trained model finds the first 2β3 correct violations confidently, then keeps reporting the same field repeatedly. This is the **classic RL fine-tuning trade-off** β GRPO heavily reinforced specific high-reward action patterns from training (Phase 2/3 episodes give +2.0 fix rewards vs Phase 1's +1.0 per violation). With more training data balanced toward Phase 1 tasks plus an explicit "don't repeat" reward signal, this would close.
|
| 304 |
+
|
| 305 |
+
**Phase 2 / Phase 3 tasks** maintain their scores in the trained adapter β the model didn't *forget* multi-service reasoning while learning the breaking-change classification.
|
| 306 |
|
| 307 |
| Phase | WandB Run | Notebook |
|
| 308 |
|---|---|---|
|
|
|
|
| 332 |
| Health check | https://pushpam14-api-contract-validator.hf.space/health |
|
| 333 |
| Trained LoRA adapter | https://huggingface.co/pushpam14/api-contract-validator-grpo-7b |
|
| 334 |
| WandB training run (300 steps) | https://wandb.ai/pushpamsubscriptions-inn/openenv-contract-guardian/runs/gch0eg3k |
|
| 335 |
+
| Training proof + full logs | [`results/TRAINING_RUN_PROOF.md`](results/TRAINING_RUN_PROOF.md) |
|
| 336 |
| Training Notebook (Colab) | [`training/grpo_colab.ipynb`](training/grpo_colab.ipynb) |
|
| 337 |
| Story + Technical Guide | [`ENTERPRISE_CONTRACT_GUARDIAN_STORY.md`](ENTERPRISE_CONTRACT_GUARDIAN_STORY.md) |
|
| 338 |
| GitHub repo | https://github.com/kumarpushpam17-personal/Hackathon |
|
| 339 |
+
| HF mini-blog writeup (separate MD in Space) | [`BLOG.md`](BLOG.md) |
|
| 340 |
+
| Trained adapter model card | https://huggingface.co/pushpam14/api-contract-validator-grpo-7b |
|
| 341 |
|
| 342 |
Quick test:
|
| 343 |
|