Hemanth Kunta commited on
Commit
2c92b66
Β·
unverified Β·
1 Parent(s): 58497f4

Update README with OpenEnv validation and usage steps

Browse files

Added detailed usage instructions and validation steps for the DataQualityEnv v2.

Files changed (1) hide show
  1. README.md +109 -1
README.md CHANGED
@@ -5,6 +5,114 @@ emoji: πŸš€
5
  colorFrom: blue
6
  colorTo: green
7
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
 
9
  ## Description
10
  DataQualityEnv v2 is a budget-constrained, confidence-scored OpenEnv environment where an AI agent performs multi-step SQL auditing and optional fix verification.
@@ -99,4 +207,4 @@ python inference.py
99
  ## Validation
100
  ```bash
101
  ./validate-submission.sh https://your-space.hf.space
102
- ```
 
5
  colorFrom: blue
6
  colorTo: green
7
  ---
8
+ OpenEnv Validation
9
+ This environment has been validated using OpenEnv:
10
+
11
+ openenv validate
12
+ # [OK] data-quality-env: Ready for multi-mode deployment
13
+
14
+ ## Description
15
+ DataQualityEnv v2 is a budget-constrained, confidence-scored OpenEnv environment where an AI agent performs multi-step SQL auditing and optional fix verification.
16
+
17
+ ## πŸš€ How to Use
18
+
19
+ ### Step 1: Reset the Environment
20
+ Click **"Reset episode"** to start a new task.
21
+
22
+ You will see:
23
+ - task description
24
+ - table schema
25
+ - step budget and query credits
26
+
27
+ ---
28
+
29
+ ### Step 2: Investigate with SQL Queries
30
+
31
+ Run SQL queries to explore the data and identify issues.
32
+
33
+ Examples:
34
+
35
+ ```sql
36
+ SELECT COUNT(*) FROM customers;
37
+
38
+ SELECT SUM(CASE WHEN email IS NULL THEN 1 ELSE 0 END) FROM customers;
39
+
40
+ SELECT customer_id, email, COUNT(*)
41
+ FROM customers
42
+ GROUP BY 1,2
43
+ HAVING COUNT(*) > 1;
44
+ πŸ’‘ Note:
45
+
46
+ Queries help you gather evidence
47
+ They may return small rewards or penalties
48
+ Final scoring does NOT happen here
49
+ Step 3: Submit Audit Report (IMPORTANT)
50
+
51
+ After analysis, submit your findings using:
52
+ Observation json: {
53
+ "null_issues": {
54
+ "email": 12,
55
+ "customer_id": 0
56
+ },
57
+ "duplicate_row_count": 15,
58
+ "near_duplicate_count": 9,
59
+ "confidence": 0.9
60
+ }
61
+ submit: {
62
+ "null_issues": {
63
+ "email": 12,
64
+ "customer_id": 0
65
+ },
66
+ "duplicate_row_count": 15,
67
+ "near_duplicate_count": 9,
68
+ "confidence": 0.9
69
+ }
70
+ πŸ“Œ This step triggers:
71
+
72
+ deterministic grading
73
+ final score calculation (0.0 – 1.0)
74
+
75
+ ⚠️ Important:
76
+
77
+ Running queries alone will NOT produce a final score.
78
+ You must submit a report.
79
+
80
+ Step 4: Fix Phase (Optional)
81
+
82
+ After submitting the report, the environment enters fix phase.
83
+
84
+ You can propose fixes using SQL:
85
+ UPDATE customers SET email = NULL WHERE email = 'UNKNOWN';
86
+ This may improve your score.
87
+
88
+ Step 5: Final Result
89
+
90
+ You will receive:
91
+
92
+ {
93
+ "value": 0.85,
94
+ "done": true
95
+ }
96
+ value β†’ final score
97
+ done β†’ task completed
98
+ πŸ€– Auto Audit Mode
99
+
100
+ You can also click "Auto audit", which:
101
+
102
+ runs multiple diagnostic SQL queries
103
+ generates a report automatically
104
+ submits it before step limit
105
+
106
+ This is the fastest way to test the system.
107
+
108
+ 🧠 Reward System
109
+ Queries may return small penalties (e.g., -0.1) for redundant or low-value actions
110
+ This encourages efficient exploration
111
+ Final score depends ONLY on the submitted report
112
+ ⚠️ Important Notes
113
+ Always submit a report before step limit
114
+ If you don't, the system may auto-submit a fallback report
115
+ Efficient querying leads to better performance
116
 
117
  ## Description
118
  DataQualityEnv v2 is a budget-constrained, confidence-scored OpenEnv environment where an AI agent performs multi-step SQL auditing and optional fix verification.
 
207
  ## Validation
208
  ```bash
209
  ./validate-submission.sh https://your-space.hf.space
210
+ ```