Spaces:
Sleeping
Sleeping
Upload folder using huggingface_hub
Browse files- README.md +38 -0
- openenv.yaml +3 -3
README.md
CHANGED
|
@@ -17,6 +17,44 @@ base_path: /web
|
|
| 17 |
|
| 18 |
---
|
| 19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
## π Introduction
|
| 21 |
|
| 22 |
Large Language Model APIs charge per token. Verbose prompts and unconstrained responses waste tokens and money. This environment trains an AI agent to **rewrite verbose prompts into concise, efficient versions** that:
|
|
|
|
| 17 |
|
| 18 |
---
|
| 19 |
|
| 20 |
+
## β οΈ Hackathon Status β Action Required
|
| 21 |
+
|
| 22 |
+
**Submission #6 failed Phase 2 validation.** All fixes have been applied and pushed. **Someone needs to resubmit from the dashboard before 12 April 2026, 11:59 PM IST.**
|
| 23 |
+
|
| 24 |
+
### What was fixed (latest commits)
|
| 25 |
+
|
| 26 |
+
| Commit | Fix |
|
| 27 |
+
|--------|-----|
|
| 28 |
+
| `c7fe490` | Fixed `openenv.yaml` β tasks had wrong format (plain strings, no `grader` field) |
|
| 29 |
+
| `3dde704` | Each task now has its own grader function (`grade_redundancy_stripping`, etc.) |
|
| 30 |
+
| `24eb57f` | Fixed fallback simulate domain responses + hard task always returns JSON |
|
| 31 |
+
|
| 32 |
+
### Bugs fixed in `server/token_optimiser_environment.py`
|
| 33 |
+
|
| 34 |
+
1. **Easy task fallback** β format check was `"brief explanation"` (never matched `"plain_brief"`), response was about machine learning instead of photosynthesis
|
| 35 |
+
2. **Medium task fallback** β response was about solar/renewable energy instead of Python vs JavaScript
|
| 36 |
+
3. **Hard task fallback** β JSON only returned when compression ratio β€ 0.6; now always returns valid JSON with all 5 required keys
|
| 37 |
+
4. **Keyword scorer** β added photosynthesis-domain keywords (plants, sunlight, co2, oxygen, etc.)
|
| 38 |
+
|
| 39 |
+
### Updated baseline (no HF_TOKEN)
|
| 40 |
+
|
| 41 |
+
| Task | Before | After fix |
|
| 42 |
+
|------|--------|-----------|
|
| 43 |
+
| Easy | 0.657 | ~0.75 |
|
| 44 |
+
| Medium | 0.673 | ~0.49 |
|
| 45 |
+
| Hard | 0.137 | ~0.60 |
|
| 46 |
+
|
| 47 |
+
> Medium score may vary with the rule-based optimizer; with a real LLM it should score higher.
|
| 48 |
+
|
| 49 |
+
### Checklist before resubmit
|
| 50 |
+
- [x] `openenv.yaml` has 3 tasks with correct `grader` fields
|
| 51 |
+
- [x] All 3 grader functions exist and are importable
|
| 52 |
+
- [x] Fallback responses match actual task domains
|
| 53 |
+
- [x] `inference.py` runs all 3 task rounds (`TASK_EVAL_ROUNDS = 3`)
|
| 54 |
+
- [ ] **Resubmit from dashboard** β https://openenv.scaler.com (or wherever the dashboard is)
|
| 55 |
+
|
| 56 |
+
---
|
| 57 |
+
|
| 58 |
## π Introduction
|
| 59 |
|
| 60 |
Large Language Model APIs charge per token. Verbose prompts and unconstrained responses waste tokens and money. This environment trains an AI agent to **rewrite verbose prompts into concise, efficient versions** that:
|
openenv.yaml
CHANGED
|
@@ -9,14 +9,14 @@ tasks:
|
|
| 9 |
name: "Redundancy Stripping"
|
| 10 |
description: "Strip politeness filler and redundancy"
|
| 11 |
difficulty: "easy"
|
| 12 |
-
grader: "server.token_optimiser_environment:grade_redundancy_stripping"
|
| 13 |
- id: "constraint_injection"
|
| 14 |
name: "Constraint Injection"
|
| 15 |
description: "Compress input AND inject exact bullet point constraints"
|
| 16 |
difficulty: "medium"
|
| 17 |
-
grader: "server.token_optimiser_environment:grade_constraint_injection"
|
| 18 |
- id: "multi_key_json_extraction"
|
| 19 |
name: "Multi-Key JSON Extraction"
|
| 20 |
description: "Force structured JSON output with 5 exact keys"
|
| 21 |
difficulty: "hard"
|
| 22 |
-
grader: "server.token_optimiser_environment:grade_multi_key_json_extraction"
|
|
|
|
| 9 |
name: "Redundancy Stripping"
|
| 10 |
description: "Strip politeness filler and redundancy"
|
| 11 |
difficulty: "easy"
|
| 12 |
+
grader: "token_optimiser.server.token_optimiser_environment:grade_redundancy_stripping"
|
| 13 |
- id: "constraint_injection"
|
| 14 |
name: "Constraint Injection"
|
| 15 |
description: "Compress input AND inject exact bullet point constraints"
|
| 16 |
difficulty: "medium"
|
| 17 |
+
grader: "token_optimiser.server.token_optimiser_environment:grade_constraint_injection"
|
| 18 |
- id: "multi_key_json_extraction"
|
| 19 |
name: "Multi-Key JSON Extraction"
|
| 20 |
description: "Force structured JSON output with 5 exact keys"
|
| 21 |
difficulty: "hard"
|
| 22 |
+
grader: "token_optimiser.server.token_optimiser_environment:grade_multi_key_json_extraction"
|