JayeshCC commited on
Commit
50a32fc
Β·
verified Β·
1 Parent(s): 4870a97

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. README.md +38 -0
  2. openenv.yaml +3 -3
README.md CHANGED
@@ -17,6 +17,44 @@ base_path: /web
17
 
18
  ---
19
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
  ## πŸ“Œ Introduction
21
 
22
  Large Language Model APIs charge per token. Verbose prompts and unconstrained responses waste tokens and money. This environment trains an AI agent to **rewrite verbose prompts into concise, efficient versions** that:
 
17
 
18
  ---
19
 
20
+ ## ⚠️ Hackathon Status β€” Action Required
21
+
22
+ **Submission #6 failed Phase 2 validation.** All fixes have been applied and pushed. **Someone needs to resubmit from the dashboard before 12 April 2026, 11:59 PM IST.**
23
+
24
+ ### What was fixed (latest commits)
25
+
26
+ | Commit | Fix |
27
+ |--------|-----|
28
+ | `c7fe490` | Fixed `openenv.yaml` β€” tasks had wrong format (plain strings, no `grader` field) |
29
+ | `3dde704` | Each task now has its own grader function (`grade_redundancy_stripping`, etc.) |
30
+ | `24eb57f` | Fixed fallback simulate domain responses + hard task always returns JSON |
31
+
32
+ ### Bugs fixed in `server/token_optimiser_environment.py`
33
+
34
+ 1. **Easy task fallback** β€” format check was `"brief explanation"` (never matched `"plain_brief"`), response was about machine learning instead of photosynthesis
35
+ 2. **Medium task fallback** β€” response was about solar/renewable energy instead of Python vs JavaScript
36
+ 3. **Hard task fallback** β€” JSON only returned when compression ratio ≀ 0.6; now always returns valid JSON with all 5 required keys
37
+ 4. **Keyword scorer** β€” added photosynthesis-domain keywords (plants, sunlight, co2, oxygen, etc.)
38
+
39
+ ### Updated baseline (no HF_TOKEN)
40
+
41
+ | Task | Before | After fix |
42
+ |------|--------|-----------|
43
+ | Easy | 0.657 | ~0.75 |
44
+ | Medium | 0.673 | ~0.49 |
45
+ | Hard | 0.137 | ~0.60 |
46
+
47
+ > Medium score may vary with the rule-based optimizer; with a real LLM it should score higher.
48
+
49
+ ### Checklist before resubmit
50
+ - [x] `openenv.yaml` has 3 tasks with correct `grader` fields
51
+ - [x] All 3 grader functions exist and are importable
52
+ - [x] Fallback responses match actual task domains
53
+ - [x] `inference.py` runs all 3 task rounds (`TASK_EVAL_ROUNDS = 3`)
54
+ - [ ] **Resubmit from dashboard** β†’ https://openenv.scaler.com (or wherever the dashboard is)
55
+
56
+ ---
57
+
58
  ## πŸ“Œ Introduction
59
 
60
  Large Language Model APIs charge per token. Verbose prompts and unconstrained responses waste tokens and money. This environment trains an AI agent to **rewrite verbose prompts into concise, efficient versions** that:
openenv.yaml CHANGED
@@ -9,14 +9,14 @@ tasks:
9
  name: "Redundancy Stripping"
10
  description: "Strip politeness filler and redundancy"
11
  difficulty: "easy"
12
- grader: "server.token_optimiser_environment:grade_redundancy_stripping"
13
  - id: "constraint_injection"
14
  name: "Constraint Injection"
15
  description: "Compress input AND inject exact bullet point constraints"
16
  difficulty: "medium"
17
- grader: "server.token_optimiser_environment:grade_constraint_injection"
18
  - id: "multi_key_json_extraction"
19
  name: "Multi-Key JSON Extraction"
20
  description: "Force structured JSON output with 5 exact keys"
21
  difficulty: "hard"
22
- grader: "server.token_optimiser_environment:grade_multi_key_json_extraction"
 
9
  name: "Redundancy Stripping"
10
  description: "Strip politeness filler and redundancy"
11
  difficulty: "easy"
12
+ grader: "token_optimiser.server.token_optimiser_environment:grade_redundancy_stripping"
13
  - id: "constraint_injection"
14
  name: "Constraint Injection"
15
  description: "Compress input AND inject exact bullet point constraints"
16
  difficulty: "medium"
17
+ grader: "token_optimiser.server.token_optimiser_environment:grade_constraint_injection"
18
  - id: "multi_key_json_extraction"
19
  name: "Multi-Key JSON Extraction"
20
  description: "Force structured JSON output with 5 exact keys"
21
  difficulty: "hard"
22
+ grader: "token_optimiser.server.token_optimiser_environment:grade_multi_key_json_extraction"