debug-env / debug_env /tasks /incident1 /seed_data.py
hugsbun's picture
feat: add SRE incident benchmark (Pivot 2 - Scenario A)
a9f2957
Raw
History Blame Contribute Delete
2.83 kB
"""
Incident 1 seed data β€” "Poisoned Database Record" scenario.
Returns the SQL statements to initialise the SQLite database and the
PagerDuty-style alert message shown to the agent on reset.
"""
_ALERT_MESSAGE = """\
ALERT [P1]: PagerDuty triggered β€” background job worker is crash-looping.
Symptoms:
* Worker thread restarting every ~0.5 s (app.log shows repeated [WORKER CRASH])
* ~15 % HTTP 500 rate on POST /checkout β€” users see "Serialization Error"
* Jobs queue depth increasing: pending jobs are not being drained
Your objective:
1. Diagnose the root cause using query_logs, curl_endpoint, and execute_sql
2. Fix the corrupted database state (update or delete the bad job row)
3. Patch worker.py to handle malformed payloads gracefully (add try/except)
4. Call restart_service to restart the server and trigger final verification
Available tools: query_logs, curl_endpoint, execute_sql, read_file, edit_file,
list_files, restart_service
"""
_DB_SEED_SQL = [
"PRAGMA journal_mode=WAL",
(
"CREATE TABLE IF NOT EXISTS jobs "
"(id INTEGER PRIMARY KEY, payload TEXT, status TEXT DEFAULT 'pending')"
),
# Completed jobs (healthy baseline)
"INSERT INTO jobs (payload, status) VALUES ('{\"item\": \"widget\", \"qty\": 5}', 'completed')",
"INSERT INTO jobs (payload, status) VALUES ('{\"item\": \"gadget\", \"qty\": 2}', 'completed')",
"INSERT INTO jobs (payload, status) VALUES ('{\"item\": \"doohickey\", \"qty\": 10}', 'completed')",
"INSERT INTO jobs (payload, status) VALUES ('{\"item\": \"thingamajig\", \"qty\": 1}', 'completed')",
"INSERT INTO jobs (payload, status) VALUES ('{\"item\": \"whatsit\", \"qty\": 7}', 'completed')",
# Valid pending job processed before the crash
"INSERT INTO jobs (payload, status) VALUES ('{\"item\": \"widget_6\", \"qty\": 3}', 'pending')",
# ── POISONED ROW ── truncated JSON; json.loads raises JSONDecodeError
"INSERT INTO jobs (payload, status) VALUES ('{\"item\": \"toxic_widget\", \"qty\":', 'pending')",
# Further pending jobs β€” never reached due to crash above
"INSERT INTO jobs (payload, status) VALUES ('{\"item\": \"widget_8\", \"qty\": 4}', 'pending')",
"INSERT INTO jobs (payload, status) VALUES ('{\"item\": \"widget_9\", \"qty\": 6}', 'pending')",
]
def get_incident1_seed_data() -> dict:
"""
Return the seed data dict for the incident1 scenario.
Keys:
alert_message (str) PagerDuty-style alert shown to the agent.
db_seed_sql (list) SQL statements to create and populate app.db.
server_entrypoint (str) Filename of the FastAPI server script.
"""
return {
"alert_message": _ALERT_MESSAGE,
"db_seed_sql": _DB_SEED_SQL,
"server_entrypoint": "buggy_server.py",
}