aastikny commited on
Commit
16e82bc
ยท
verified ยท
1 Parent(s): 8c6d4ba

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -10
README.md CHANGED
@@ -7,21 +7,32 @@ sdk: docker
7
  app_file: app.py
8
  pinned: false
9
  ---
10
- # SQLite Rescue Environment
11
 
12
- ## Description
13
- A data engineering environment where an AI agent must clean, normalize, and manipulate a messy SQLite database using raw SQL queries. This simulates the real-world task of data cleaning and database refactoring.
14
 
15
- ## Action & Observation Spaces
 
 
 
 
 
 
 
 
 
 
16
  * **Action Space:** The agent submits an SQL `query` (string) to execute, and a `submit` (boolean) flag when they are ready for their final database state to be graded.
17
- * **Observation Space:** The environment returns the current `schema_info` (string), `rows_affected` (int), any SQL execution `error` (string), and a `query_result` (list of dicts) if the action was a SELECT query.
18
 
19
- ## Tasks
20
  1. **easy_data_cleaning:** Clean inconsistent dates and trailing whitespaces in a single table.
21
  2. **medium_schema_normalization:** Split a denormalized monolithic table into two related tables with a foreign key.
22
  3. **hard_complex_reconciliation:** Write a complex query/view to generate a financial reconciliation report.
23
 
24
- ## Setup Instructions
25
- 1. Install dependencies: `pip install -r requirements.txt` (or via pyproject.toml)
26
- 2. Generate the starting databases: `python generate_templates.py`
27
- 3. Run the baseline: `python inference.py`
 
 
 
 
7
  app_file: app.py
8
  pinned: false
9
  ---
 
10
 
11
+ # ๐Ÿ—„๏ธ SQLite Rescue Environment
 
12
 
13
+ ## ๐Ÿ“– What is this project?
14
+ This is an **Agentic Evaluation Environment** designed to test how well Large Language Models (LLMs) can perform real-world data engineering tasks.
15
+
16
+ Instead of just answering trivia, an AI agent is dropped into a messy SQLite database. Its job is to explore the schema, write raw SQL queries to clean and normalize the data, and submit its final work. The environment acts as a secure sandbox, executing the queries, providing feedback (or errors) back to the agent, and automatically grading the AI's final database state.
17
+
18
+ ## โš™๏ธ How It Works
19
+ 1. **The Sandbox Manager:** When the `/reset` API endpoint is called, the environment creates a fresh, isolated temporary database from a messy template.
20
+ 2. **The Observation Loop:** The AI agent sends raw SQL queries as actions. The environment executes them safely and returns the results, row counts, or syntax errors, allowing the agent to "see" the database state.
21
+ 3. **The Automated Judge:** When the agent sets the `submit` flag to `True`, custom Python grader scripts analyze the final database state and assign a reward score (0.0 to 1.0) based on how accurately the data was fixed.
22
+
23
+ ## ๐Ÿ•น๏ธ Action & Observation Spaces
24
  * **Action Space:** The agent submits an SQL `query` (string) to execute, and a `submit` (boolean) flag when they are ready for their final database state to be graded.
25
+ * **Observation Space:** The environment returns the current `schema_info` (string), `rows_affected` (int), any SQL execution `error` (string), and a `query_result` (list of dicts) if the action was a `SELECT` query.
26
 
27
+ ## ๐Ÿ“‹ Evaluation Tasks
28
  1. **easy_data_cleaning:** Clean inconsistent dates and trailing whitespaces in a single table.
29
  2. **medium_schema_normalization:** Split a denormalized monolithic table into two related tables with a foreign key.
30
  3. **hard_complex_reconciliation:** Write a complex query/view to generate a financial reconciliation report.
31
 
32
+ ## ๐Ÿš€ Setup & Execution
33
+ This project uses modern Python packaging via `pyproject.toml`, eliminating the need for a legacy `requirements.txt`.
34
+
35
+ **1. Install dependencies:**
36
+ ```bash
37
+ pip install .
38
+ # Or, if you are using uv: uv pip install .