Download SPEC.md from MF180/autonomous-bug-fixing-agent: direct link, hf CLI and curl.
- Browser
- Download file 19.3 kB
-
https://huggingface.co/MF180/autonomous-bug-fixing-agent/resolve/main/SPEC.md
- Command line
-
hf download hf://MF180/autonomous-bug-fixing-agent/SPEC.md
-
curl -L -o SPEC.md https://huggingface.co/MF180/autonomous-bug-fixing-agent/resolve/main/SPEC.md
# Autonomous Bug-Fixing Agent  Architecture Spec
Status: DRAFT v0.1  for review. No implementation exists yet; every file under src/ is a stub.
1. Goal & Flow
Input: a GitHub issue / bug description + a local repo path. Output: a reviewed, human-approved commit (and optionally a PR) containing a fix and a regression test.
flowchart TD
A[ingest_issue] --> B[inspect_repo]
B --> C[write_repro_test]
C --> D[run_repro_test]
D -->|fails as expected| E[propose_fix]
D -->|passes: not reproduced, retries left| C
D -->|passes: retries exhausted| X[escalate]
E --> F[apply_fix]
F --> G[run_tests]
G -->|pass| H[human_approval -- interrupt]
G -->|fail, iteration < 8| I[analyze_failure]
G -->|fail, iteration >= 8| X
I --> E
H -->|approve| J[commit_and_pr]
H -->|reject + feedback| I
H -->|reject, abort| X
J --> Z((END))
X --> Z
Every LLM-calling node passes through the context pruner and the Groq rate-limit wrapper (section 6).
2. State Schema (src/agent/state.py)
The graph state is a TypedDict (LangGraph requirement for reducers). Structured sub-objects are Pydantic v2 models, so they are validated at node boundaries and serialize cleanly through checkpointers (model_dump(mode="json")).
2.1 Pydantic models
| Model | Fields |
|---|---|
IssueInput |
title: str, body: str, url: str | None, number: int | None, repo_path: Path |
RepoContext |
tree: str (truncated listing), relevant_files: dict[str, str] (path → content snippet), test_command: str, language: str |
ReproTest |
path: str, content: str, framework: Literal["pytest"], confirmed_failing: bool |
FilePatch |
path: str, unified_diff: str |
FixAttempt |
iteration: int, rationale: str, patches: list[FilePatch], result: SandboxResult | None |
SandboxResult |
exit_code: int, stdout: str, stderr: str, timed_out: bool, duration_s: float, truncated: bool (shared with the sandbox interface; defined in docker_runner.py, re-exported by state.py) |
ApprovalDecision |
approved: bool, feedback: str | None, reviewer: str | None, decided_at: datetime |
RunBudget |
max_iterations: int = 8, max_repro_attempts: int = 2, max_context_tokens: int = 6000, max_tool_output_chars: int = 4000 |
2.2 AgentStatus (StrEnum)
INGESTED, INSPECTED, REPRO_WRITTEN, REPRO_CONFIRMED, FIXING, TESTS_PASSING, AWAITING_APPROVAL, APPROVED, COMMITTED, ESCALATED, REJECTED.
2.3 AgentState (TypedDict)
class AgentState(TypedDict, total=False):
issue: IssueInput
repo: RepoContext
repro_test: ReproTest
attempts: Annotated[list[FixAttempt], merge_attempts] # history, upserted by iteration
messages: Annotated[list[AnyMessage], add_messages] # LLM conversation (pruned)
iteration: int # fix iterations consumed (0..max_iterations)
repro_attempts: int # repro-test generations consumed
budget: RunBudget
status: AgentStatus
approval: ApprovalDecision | None
branch: str | None
commit_sha: str | None
pr_url: str | None
error: str | None # terminal failure reason, set by escalate
Rules:
- Nodes return partial updates only; they never mutate the input state.
attemptsis full history for the audit trail. Its reducermerge_attemptsupserts byiteration, sorun_testscan return an updated copy of the latest attempt with itsSandboxResultattached (a plain append reducer cannot update an entry). The LLM never sees it raw; the pruner builds a compact summary (section 6).messagesis the only field the pruner rewrites.
3. Node Functions (src/agent/nodes.py)
All nodes: (state: AgentState, config: RunnableConfig) -> dict (partial update). Dependencies (LLM, sandbox, git tools) come from config["configurable"], so tests can inject fakes.
| Node | Responsibility | Reads | Writes |
|---|---|---|---|
ingest_issue |
Validate/normalize the issue (parse a GitHub URL via API if given, else use the raw text); init budget and counters. | issue |
issue, budget, iteration=0, status |
inspect_repo |
Build the repo tree, grep for symbols/stack traces named in the issue, read the top-N relevant files (size-capped), detect the test command. Uses file_tools. |
issue |
repo, status |
write_repro_test |
LLM writes a minimal failing pytest test; saved into the working tree. | issue, repo, messages |
repro_test, repro_attempts+1, messages |
run_repro_test |
Run the repro test in the sandbox. We expect failure. A pass means the bug was not reproduced. | repro_test |
repro_test.confirmed_failing, status, messages |
propose_fix |
LLM returns structured output (FixProposal: rationale + patches) via with_structured_output. |
pruned context | attempts (+1, result=None), messages |
apply_fix |
Apply unified diffs in the working tree (file_tools.apply_patch); on patch failure, record the error and loop back without spending a sandbox run. |
attempts[-1] |
working tree, attempts[-1] error |
run_tests |
Run the repro test plus the existing suite in the sandbox; attach SandboxResult to the latest attempt; iteration += 1. |
repo.test_command |
attempts[-1].result, iteration, status |
analyze_failure |
LLM reads the (truncated) failing output + human feedback if any; produces diagnosis for the next propose_fix. |
attempts[-1], approval |
messages |
human_approval |
Builds the review payload (diff, test result, rationale) and calls langgraph.types.interrupt(payload). Resumed with Command(resume=ApprovalDecision-dict). The only path to commit_and_pr. |
attempts, repro_test |
approval, status |
commit_and_pr |
git_tools: create branch agent/fix-<issue#>, stage, commit, optionally push + open PR via gh/API. Runs only on approved state. |
approval, issue |
branch, commit_sha, pr_url, status=COMMITTED |
escalate |
Terminal failure: write a report (what was tried, last output) and set error. Never commits. |
all | error, status=ESCALATED |
4. Conditional Edge Routing (src/agent/graph.py)
Pure functions of state (no I/O), so they are trivially unit-testable.
def route_after_repro(state) -> Literal["propose_fix", "write_repro_test", "escalate"]:
if state["repro_test"].confirmed_failing: return "propose_fix"
if state["repro_attempts"] < state["budget"].max_repro_attempts: return "write_repro_test"
return "escalate"
def route_after_apply(state) -> Literal["run_tests", "analyze_failure"]:
return "analyze_failure" if state["attempts"][-1].patch_error else "run_tests"
def route_after_tests(state) -> Literal["human_approval", "analyze_failure", "escalate"]:
if state["attempts"][-1].result.exit_code == 0: return "human_approval"
if state["iteration"] < state["budget"].max_iterations: return "analyze_failure"
return "escalate"
def route_after_approval(state) -> Literal["commit_and_pr", "analyze_failure", "escalate"]:
a = state["approval"]
if a.approved: return "commit_and_pr"
if a.feedback and state["iteration"] < state["budget"].max_iterations:
return "analyze_failure" # reviewer asked for changes
return "escalate"
Invariants (each gets a test):
iterationnever exceedsmax_iterations(8). LangGraphrecursion_limitis additionally set to a safe bound (about 4× nodes × iterations) as a hard backstop.commit_and_prhas exactly one inbound edge: fromhuman_approvalviaroute_after_approval.- Timeouts and
SandboxResult.timed_outcount as failures, never as passes.
Checkpointing & HITL
build_graph(checkpointer: BaseCheckpointSaver | None = None)compiles withMemorySaver()by default.get_checkpointer(settings)factory:MemorySaverfor dev;PostgresSaver.from_conn_string(DATABASE_URL)whenCHECKPOINTER=postgres(extra dependencypostgres;.setup()called once at startup).- State must stay JSON-serializable for Postgres. This is why the models are Pydantic and
Pathis serialized asstr. - Runs are keyed by
thread_id. A paused run is resumed withgraph.invoke(Command(resume=decision), config).
5. Docker Sandbox Interface (src/sandbox/docker_runner.py)
Library: docker (docker-py). The runner is a long-lived container per agent run (so installed state and edits persist across iterations). Commands run via exec_run.
class SandboxConfig(BaseModel):
image: str = "bugfix-agent-sandbox:latest" # built from src/sandbox/Dockerfile
workdir: str = "/workspace"
mem_limit: str = "512m"
nano_cpus: int = 1_000_000_000 # 1 CPU
pids_limit: int = 256
network_disabled: bool = True # default-deny network
default_timeout_s: int = 60
max_output_bytes: int = 64_000 # stdout/stderr each, tail-preserved
user: str = "sandbox" # non-root
class SandboxResult(BaseModel):
exit_code: int; stdout: str; stderr: str
timed_out: bool; duration_s: float; truncated: bool
class SandboxError(Exception): ... # daemon unavailable, image missing, container died
class DockerSandbox:
def __init__(self, config: SandboxConfig | None = None, client: DockerClient | None = None): ...
def start(self, host_repo_path: Path) -> None # create container, mount working copy
def run(self, command: str, timeout_s: int | None = None) -> SandboxResult # bash -c
def run_tests(self, test_command: str, timeout_s: int | None = None) -> SandboxResult
def write_file(self, path: str, content: str) -> None
def read_file(self, path: str) -> str
def stop(self) -> None # idempotent; force-remove container
def __enter__(self) / __exit__(...) # guarantees cleanup
@staticmethod
def is_available() -> bool # daemon reachable
Design decisions:
- Isolation: non-root user,
cap_drop=["ALL"],security_opt=["no-new-privileges"], memory, CPU and PID limits, network off by default. Dependencies are pre-baked into the image. If a repo needs installs, there is an explicit opt-innetwork_disabled=Falsesetup phase. - Working copy, not the original repo: the agent works on a
git worktreeor temp copy of the repo and bind-mounts it at/workspace. The user's checkout is untouched until approval. - Timeouts: enforced inside the container with
timeout <n> bash -c .... Exit code 124 maps totimed_out=True. This avoids orphaned processes, which a host-side timeout onexec_runwould leave behind. - Output cap: stdout/stderr are tail-truncated to
max_output_bytes(failures appear at the end of pytest output). The state/prompt layer truncates again to about 4k chars for the LLM. - Cleanup: the context manager and
stop()always remove the container, including on exceptions.
6. Groq Free-Tier Guardrails
| Concern | Mechanism | Location |
|---|---|---|
| 429 / transient errors | tenacity @retry(wait=wait_random_exponential(multiplier=2, max=60), stop=stop_after_attempt(6), retry=retry_if_exception_type(RateLimitError, APIConnectionError, ...)). Honors the retry-after header when present. |
src/agent/llm.py (added in a later phase) |
| Runaway loops | max_iterations = 8 in RunBudget + routing invariants + LangGraph recursion_limit. |
state, graph |
| Context growth | prune_messages() runs before every LLM call: (1) always keep system prompt, issue, and latest repro test; (2) replace old attempts with a one-line summary each (iter 3: patched foo.py, test X failed: AssertionError); (3) keep only the latest sandbox output, truncated to max_tool_output_chars; (4) final guard via langchain_core.messages.trim_messages(max_tokens=max_context_tokens, strategy="last"). |
src/agent/nodes.py / helper |
| TPM bursts | Lightweight client-side pacing (min interval between calls) plus capped max_tokens on completions. |
llm.py |
| Model | ChatGroq(model="llama-3.3-70b-versatile", temperature=0), GROQ_API_KEY from env. |
llm.py |
7. Project Layout
.
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ SPEC.md
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ pyproject.toml
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ .env.example
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ .gitignore
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ src/
ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ agent/
ââ€â€š ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ state.py # models + AgentState (phase 1)
ââ€â€š ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ graph.py # build_graph, routers, checkpointer (phase 3)
ââ€â€š ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ nodes.py # node functions + pruner (phase 3)
ââ€â€š ââ€â€š ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ prompts.py # system / task prompts (phase 3)
ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ sandbox/
ââ€â€š ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ docker_runner.py # DockerSandbox (phase 1)
ââ€â€š ââ€â€š ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ Dockerfile # python:3.12-slim + pytest, non-root user
ââ€â€š ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ tools/
ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ file_tools.py # read/list/grep/apply_patch, path-jail (phase 2)
ââ€â€š ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ git_tools.py # branch/commit/push/PR (phase 2)
ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ tests/
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ conftest.py # `docker` marker, skip if daemon down
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ unit/
ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ test_state.py (phase 1)
ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ test_docker_runner.py (phase 1; integration-marked)
ââ€â€š ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ test_routing.py (phase 3)
ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ fixtures/
ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ buggy_repo/ # tiny repo with a known bug + expected repro test
8. Phases
- Phase 1:
state.py,docker_runner.py, Dockerfile, with unit tests that run real containers (markeddocker, auto-skipped if there is no daemon). - Phase 2:
file_tools.py(path-jailed to the repo root),git_tools.py. - Phase 3:
prompts.py,nodes.py,graph.py,llm.py, routing tests with a fake LLM. - Phase 4: End-to-end run against
fixtures/buggy_repo, CLI entry point, README. - Phase 5: Streamlit UI and deployment (section 11). Requires the sandbox backend abstraction (section 10), which is built right after Phase 2.
9. Open Questions for Review
- Language scope: Python/pytest only for v1? (This spec assumes yes.)
- PR creation:
ghCLI vs. PyGithub / REST. Spec leansghfor fewer dependencies, with push + PR optional (commit-only by default). - Repo isolation:
git worktree(requires a git repo) vs. plain temp-dir copy. Spec leans worktree. - Network in the sandbox: off by default with pre-baked deps. OK, or do you want an install phase with network enabled?
- LLM client:
langchain-groq(ChatGroq) is assumed over a raw OpenAI-compatible client.
10. Sandbox Backends (added for Streamlit deployment)
Streamlit Community Cloud has no Docker daemon, so the sandbox becomes pluggable.
src/sandbox/base.py: aSandboxProtocolwithstart(path),run,run_tests,write_file,read_file,stop, and context-manager support.SandboxResultandSandboxErrormove here and are re-exported fromdocker_runner.py, so nothing else changes.DockerSandbox(existing): full isolation (section 5). The default for local use and any host with Docker.LocalSandbox(src/sandbox/local_runner.py): runs commands as subprocesses in the working-copy directory, with the same timeout and tail-truncated output contract.- Weaker isolation, and the UI says so. It does not block the network, jail the filesystem, or cap memory.
- It scrubs the environment:
GROQ_API_KEY,GITHUB_TOKEN,DATABASE_URL, and anything matching*KEY*,*TOKEN*,*SECRET*, so LLM-generated code cannot read secrets (Streamlit secrets are exposed as environment variables). - The directory jail is enforced in
write_file/read_fileonly. Restrict it to demo-sized repos cloned into a temp directory.
create_sandbox(settings)picks the backend fromSANDBOX_BACKEND=docker|local|auto.autouses Docker if the daemon is reachable, otherwiseLocalSandboxwith a logged warning. Nodes depend only on theSandboxprotocol.- Side benefit: agent nodes and the graph can be tested end to end without Docker.
11. Streamlit App (Phase 5)
- Entry point:
src/ui/streamlit_app.py; extra dependency groupui = ["streamlit"]. - Inputs: repo (GitHub URL, cloned to a temp dir with a size cap) and an issue URL or pasted text. The Groq key comes from
st.secretsor a session-only password field. It is never logged or written to disk. - Run loop:
graph.stream(..., thread_id=st.session_state.thread_id). Each node shows as a step inst.status, with the repro test, diffs and sandbox output in expanders. - HITL: when the graph returns an interrupt, the UI renders the diff and test results with Approve and Reject + feedback buttons. The click calls
graph.invoke(Command(resume=...), config).commit_and_prstill runs only after this. - Commit/PR on the cloud: the default is "download patch / commit message". Pushing a branch and opening a PR is opt-in and uses a user-supplied GitHub token held in session state only.
- Persistence:
MemorySaveris per process, so state is lost on app restart or sleep. UsePostgresSaver(CHECKPOINTER=postgres) if runs must survive restarts. - Abuse and cost: the 8-iteration cap, per-session run limit and token-aware pruning bound Groq usage. A public app should either ask users for their own key or be restricted. A shared key invites quota exhaustion.
- Deploy: the repo needs
requirements.txt(orpyproject.toml) withstreamlit, secrets set in the Community Cloud UI (.streamlit/secrets.tomlis gitignored locally), andSANDBOX_BACKEND=localthere. Free-tier apps have limited RAM and sleep when idle.