File size: 19,337 Bytes
eb60226 26228ef eb60226 26228ef eb60226 26228ef eb60226 26228ef eb60226 26228ef eb60226 26228ef 96b8c55 26228ef 96b8c55 26228ef eb60226 26228ef 96b8c55 26228ef 96b8c55 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 | # Autonomous Bug-Fixing Agent  Architecture Spec
Status: **DRAFT v0.1  for review.** No implementation exists yet; every file under `src/` is a stub.
## 1. Goal & Flow
Input: a GitHub issue / bug description + a local repo path.
Output: a reviewed, human-approved commit (and optionally a PR) containing a fix and a regression test.
```mermaid
flowchart TD
A[ingest_issue] --> B[inspect_repo]
B --> C[write_repro_test]
C --> D[run_repro_test]
D -->|fails as expected| E[propose_fix]
D -->|passes: not reproduced, retries left| C
D -->|passes: retries exhausted| X[escalate]
E --> F[apply_fix]
F --> G[run_tests]
G -->|pass| H[human_approval -- interrupt]
G -->|fail, iteration < 8| I[analyze_failure]
G -->|fail, iteration >= 8| X
I --> E
H -->|approve| J[commit_and_pr]
H -->|reject + feedback| I
H -->|reject, abort| X
J --> Z((END))
X --> Z
```
Every LLM-calling node passes through the **context pruner** and the **Groq rate-limit wrapper** (section 6).
## 2. State Schema (`src/agent/state.py`)
The graph state is a `TypedDict` (LangGraph requirement for reducers). Structured sub-objects are **Pydantic v2** models, so they are validated at node boundaries and serialize cleanly through checkpointers (`model_dump(mode="json")`).
### 2.1 Pydantic models
| Model | Fields |
|---|---|
| `IssueInput` | `title: str`, `body: str`, `url: str \| None`, `number: int \| None`, `repo_path: Path` |
| `RepoContext` | `tree: str` (truncated listing), `relevant_files: dict[str, str]` (path → content snippet), `test_command: str`, `language: str` |
| `ReproTest` | `path: str`, `content: str`, `framework: Literal["pytest"]`, `confirmed_failing: bool` |
| `FilePatch` | `path: str`, `unified_diff: str` |
| `FixAttempt` | `iteration: int`, `rationale: str`, `patches: list[FilePatch]`, `result: SandboxResult \| None` |
| `SandboxResult` | `exit_code: int`, `stdout: str`, `stderr: str`, `timed_out: bool`, `duration_s: float`, `truncated: bool` (shared with the sandbox interface; defined in `docker_runner.py`, re-exported by `state.py`) |
| `ApprovalDecision` | `approved: bool`, `feedback: str \| None`, `reviewer: str \| None`, `decided_at: datetime` |
| `RunBudget` | `max_iterations: int = 8`, `max_repro_attempts: int = 2`, `max_context_tokens: int = 6000`, `max_tool_output_chars: int = 4000` |
### 2.2 `AgentStatus` (StrEnum)
`INGESTED`, `INSPECTED`, `REPRO_WRITTEN`, `REPRO_CONFIRMED`, `FIXING`, `TESTS_PASSING`, `AWAITING_APPROVAL`, `APPROVED`, `COMMITTED`, `ESCALATED`, `REJECTED`.
### 2.3 `AgentState` (TypedDict)
```python
class AgentState(TypedDict, total=False):
issue: IssueInput
repo: RepoContext
repro_test: ReproTest
attempts: Annotated[list[FixAttempt], merge_attempts] # history, upserted by iteration
messages: Annotated[list[AnyMessage], add_messages] # LLM conversation (pruned)
iteration: int # fix iterations consumed (0..max_iterations)
repro_attempts: int # repro-test generations consumed
budget: RunBudget
status: AgentStatus
approval: ApprovalDecision | None
branch: str | None
commit_sha: str | None
pr_url: str | None
error: str | None # terminal failure reason, set by escalate
```
Rules:
- Nodes return **partial updates only**; they never mutate the input state.
- `attempts` is full history for the audit trail. Its reducer `merge_attempts` upserts by `iteration`, so `run_tests` can return an updated copy of the latest attempt with its `SandboxResult` attached (a plain append reducer cannot update an entry). The LLM never sees it raw; the pruner builds a compact summary (section 6).
- `messages` is the only field the pruner rewrites.
## 3. Node Functions (`src/agent/nodes.py`)
All nodes: `(state: AgentState, config: RunnableConfig) -> dict` (partial update). Dependencies (LLM, sandbox, git tools) come from `config["configurable"]`, so tests can inject fakes.
| Node | Responsibility | Reads | Writes |
|---|---|---|---|
| `ingest_issue` | Validate/normalize the issue (parse a GitHub URL via API if given, else use the raw text); init budget and counters. | `issue` | `issue`, `budget`, `iteration=0`, `status` |
| `inspect_repo` | Build the repo tree, grep for symbols/stack traces named in the issue, read the top-N relevant files (size-capped), detect the test command. Uses `file_tools`. | `issue` | `repo`, `status` |
| `write_repro_test` | LLM writes a minimal failing pytest test; saved into the working tree. | `issue`, `repo`, `messages` | `repro_test`, `repro_attempts+1`, `messages` |
| `run_repro_test` | Run the repro test in the sandbox. We **expect failure**. A pass means the bug was not reproduced. | `repro_test` | `repro_test.confirmed_failing`, `status`, `messages` |
| `propose_fix` | LLM returns structured output (`FixProposal`: rationale + patches) via `with_structured_output`. | pruned context | `attempts` (+1, result=None), `messages` |
| `apply_fix` | Apply unified diffs in the working tree (`file_tools.apply_patch`); on patch failure, record the error and loop back without spending a sandbox run. | `attempts[-1]` | working tree, `attempts[-1]` error |
| `run_tests` | Run the repro test plus the existing suite in the sandbox; attach `SandboxResult` to the latest attempt; `iteration += 1`. | `repo.test_command` | `attempts[-1].result`, `iteration`, `status` |
| `analyze_failure` | LLM reads the (truncated) failing output + human feedback if any; produces diagnosis for the next `propose_fix`. | `attempts[-1]`, `approval` | `messages` |
| `human_approval` | Builds the review payload (diff, test result, rationale) and calls `langgraph.types.interrupt(payload)`. Resumed with `Command(resume=ApprovalDecision-dict)`. **The only path to `commit_and_pr`.** | `attempts`, `repro_test` | `approval`, `status` |
| `commit_and_pr` | `git_tools`: create branch `agent/fix-<issue#>`, stage, commit, optionally push + open PR via `gh`/API. Runs only on approved state. | `approval`, `issue` | `branch`, `commit_sha`, `pr_url`, `status=COMMITTED` |
| `escalate` | Terminal failure: write a report (what was tried, last output) and set `error`. Never commits. | all | `error`, `status=ESCALATED` |
## 4. Conditional Edge Routing (`src/agent/graph.py`)
Pure functions of state (no I/O), so they are trivially unit-testable.
```python
def route_after_repro(state) -> Literal["propose_fix", "write_repro_test", "escalate"]:
if state["repro_test"].confirmed_failing: return "propose_fix"
if state["repro_attempts"] < state["budget"].max_repro_attempts: return "write_repro_test"
return "escalate"
def route_after_apply(state) -> Literal["run_tests", "analyze_failure"]:
return "analyze_failure" if state["attempts"][-1].patch_error else "run_tests"
def route_after_tests(state) -> Literal["human_approval", "analyze_failure", "escalate"]:
if state["attempts"][-1].result.exit_code == 0: return "human_approval"
if state["iteration"] < state["budget"].max_iterations: return "analyze_failure"
return "escalate"
def route_after_approval(state) -> Literal["commit_and_pr", "analyze_failure", "escalate"]:
a = state["approval"]
if a.approved: return "commit_and_pr"
if a.feedback and state["iteration"] < state["budget"].max_iterations:
return "analyze_failure" # reviewer asked for changes
return "escalate"
```
Invariants (each gets a test):
1. `iteration` never exceeds `max_iterations` (8). LangGraph `recursion_limit` is additionally set to a safe bound (about 4× nodes × iterations) as a hard backstop.
2. `commit_and_pr` has exactly one inbound edge: from `human_approval` via `route_after_approval`.
3. Timeouts and `SandboxResult.timed_out` count as failures, never as passes.
### Checkpointing & HITL
- `build_graph(checkpointer: BaseCheckpointSaver | None = None)` compiles with `MemorySaver()` by default.
- `get_checkpointer(settings)` factory: `MemorySaver` for dev; `PostgresSaver.from_conn_string(DATABASE_URL)` when `CHECKPOINTER=postgres` (extra dependency `postgres`; `.setup()` called once at startup).
- State must stay JSON-serializable for Postgres. This is why the models are Pydantic and `Path` is serialized as `str`.
- Runs are keyed by `thread_id`. A paused run is resumed with `graph.invoke(Command(resume=decision), config)`.
## 5. Docker Sandbox Interface (`src/sandbox/docker_runner.py`)
Library: `docker` (docker-py). The runner is a **long-lived container per agent run** (so installed state and edits persist across iterations). Commands run via `exec_run`.
```python
class SandboxConfig(BaseModel):
image: str = "bugfix-agent-sandbox:latest" # built from src/sandbox/Dockerfile
workdir: str = "/workspace"
mem_limit: str = "512m"
nano_cpus: int = 1_000_000_000 # 1 CPU
pids_limit: int = 256
network_disabled: bool = True # default-deny network
default_timeout_s: int = 60
max_output_bytes: int = 64_000 # stdout/stderr each, tail-preserved
user: str = "sandbox" # non-root
class SandboxResult(BaseModel):
exit_code: int; stdout: str; stderr: str
timed_out: bool; duration_s: float; truncated: bool
class SandboxError(Exception): ... # daemon unavailable, image missing, container died
class DockerSandbox:
def __init__(self, config: SandboxConfig | None = None, client: DockerClient | None = None): ...
def start(self, host_repo_path: Path) -> None # create container, mount working copy
def run(self, command: str, timeout_s: int | None = None) -> SandboxResult # bash -c
def run_tests(self, test_command: str, timeout_s: int | None = None) -> SandboxResult
def write_file(self, path: str, content: str) -> None
def read_file(self, path: str) -> str
def stop(self) -> None # idempotent; force-remove container
def __enter__(self) / __exit__(...) # guarantees cleanup
@staticmethod
def is_available() -> bool # daemon reachable
```
Design decisions:
- **Isolation:** non-root user, `cap_drop=["ALL"]`, `security_opt=["no-new-privileges"]`, memory, CPU and PID limits, network off by default. Dependencies are pre-baked into the image. If a repo needs installs, there is an explicit opt-in `network_disabled=False` setup phase.
- **Working copy, not the original repo:** the agent works on a `git worktree` or temp copy of the repo and bind-mounts it at `/workspace`. The user's checkout is untouched until approval.
- **Timeouts:** enforced inside the container with `timeout <n> bash -c ...`. Exit code 124 maps to `timed_out=True`. This avoids orphaned processes, which a host-side timeout on `exec_run` would leave behind.
- **Output cap:** stdout/stderr are tail-truncated to `max_output_bytes` (failures appear at the end of pytest output). The state/prompt layer truncates again to about 4k chars for the LLM.
- **Cleanup:** the context manager and `stop()` always remove the container, including on exceptions.
## 6. Groq Free-Tier Guardrails
| Concern | Mechanism | Location |
|---|---|---|
| 429 / transient errors | `tenacity` `@retry(wait=wait_random_exponential(multiplier=2, max=60), stop=stop_after_attempt(6), retry=retry_if_exception_type(RateLimitError, APIConnectionError, ...))`. Honors the `retry-after` header when present. | `src/agent/llm.py` (added in a later phase) |
| Runaway loops | `max_iterations = 8` in `RunBudget` + routing invariants + LangGraph `recursion_limit`. | state, graph |
| Context growth | `prune_messages()` runs before every LLM call: (1) always keep system prompt, issue, and latest repro test; (2) replace old attempts with a one-line summary each (`iter 3: patched foo.py, test X failed: AssertionError`); (3) keep only the latest sandbox output, truncated to `max_tool_output_chars`; (4) final guard via `langchain_core.messages.trim_messages(max_tokens=max_context_tokens, strategy="last")`. | `src/agent/nodes.py` / helper |
| TPM bursts | Lightweight client-side pacing (min interval between calls) plus capped `max_tokens` on completions. | `llm.py` |
| Model | `ChatGroq(model="llama-3.3-70b-versatile", temperature=0)`, `GROQ_API_KEY` from env. | `llm.py` |
## 7. Project Layout
```
.
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ SPEC.md
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ pyproject.toml
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ .env.example
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ .gitignore
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ src/
ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ agent/
ââ€â€š ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ state.py # models + AgentState (phase 1)
ââ€â€š ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ graph.py # build_graph, routers, checkpointer (phase 3)
ââ€â€š ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ nodes.py # node functions + pruner (phase 3)
ââ€â€š ââ€â€š ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ prompts.py # system / task prompts (phase 3)
ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ sandbox/
ââ€â€š ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ docker_runner.py # DockerSandbox (phase 1)
ââ€â€š ââ€â€š ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ Dockerfile # python:3.12-slim + pytest, non-root user
ââ€â€š ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ tools/
ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ file_tools.py # read/list/grep/apply_patch, path-jail (phase 2)
ââ€â€š ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ git_tools.py # branch/commit/push/PR (phase 2)
ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ tests/
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ conftest.py # `docker` marker, skip if daemon down
ââ€Å“ââ€â‚¬Ã¢â€â‚¬ unit/
ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ test_state.py (phase 1)
ââ€â€š ââ€Å“ââ€â‚¬Ã¢â€â‚¬ test_docker_runner.py (phase 1; integration-marked)
ââ€â€š ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ test_routing.py (phase 3)
ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ fixtures/
ââ€â€Ã¢â€â‚¬Ã¢â€â‚¬ buggy_repo/ # tiny repo with a known bug + expected repro test
```
## 8. Phases
1. **Phase 1:** `state.py`, `docker_runner.py`, Dockerfile, with unit tests that run real containers (marked `docker`, auto-skipped if there is no daemon).
2. **Phase 2:** `file_tools.py` (path-jailed to the repo root), `git_tools.py`.
3. **Phase 3:** `prompts.py`, `nodes.py`, `graph.py`, `llm.py`, routing tests with a fake LLM.
4. **Phase 4:** End-to-end run against `fixtures/buggy_repo`, CLI entry point, README.
5. **Phase 5:** Streamlit UI and deployment (section 11). Requires the sandbox backend abstraction (section 10), which is built right after Phase 2.
## 9. Open Questions for Review
1. **Language scope:** Python/pytest only for v1? (This spec assumes yes.)
2. **PR creation:** `gh` CLI vs. PyGithub / REST. Spec leans `gh` for fewer dependencies, with push + PR optional (commit-only by default).
3. **Repo isolation:** `git worktree` (requires a git repo) vs. plain temp-dir copy. Spec leans worktree.
4. **Network in the sandbox:** off by default with pre-baked deps. OK, or do you want an install phase with network enabled?
5. **LLM client:** `langchain-groq` (`ChatGroq`) is assumed over a raw OpenAI-compatible client.
## 10. Sandbox Backends (added for Streamlit deployment)
Streamlit Community Cloud has no Docker daemon, so the sandbox becomes pluggable.
- `src/sandbox/base.py`: a `Sandbox` `Protocol` with `start(path)`, `run`, `run_tests`, `write_file`, `read_file`, `stop`, and context-manager support. `SandboxResult` and `SandboxError` move here and are re-exported from `docker_runner.py`, so nothing else changes.
- `DockerSandbox` (existing): full isolation (section 5). The default for local use and any host with Docker.
- `LocalSandbox` (`src/sandbox/local_runner.py`): runs commands as subprocesses in the working-copy directory, with the same timeout and tail-truncated output contract.
- **Weaker isolation, and the UI says so.** It does not block the network, jail the filesystem, or cap memory.
- It scrubs the environment: `GROQ_API_KEY`, `GITHUB_TOKEN`, `DATABASE_URL`, and anything matching `*KEY*`, `*TOKEN*`, `*SECRET*`, so LLM-generated code cannot read secrets (Streamlit secrets are exposed as environment variables).
- The directory jail is enforced in `write_file`/`read_file` only. Restrict it to demo-sized repos cloned into a temp directory.
- `create_sandbox(settings)` picks the backend from `SANDBOX_BACKEND=docker|local|auto`. `auto` uses Docker if the daemon is reachable, otherwise `LocalSandbox` with a logged warning. Nodes depend only on the `Sandbox` protocol.
- Side benefit: agent nodes and the graph can be tested end to end without Docker.
## 11. Streamlit App (Phase 5)
- **Entry point:** `src/ui/streamlit_app.py`; extra dependency group `ui = ["streamlit"]`.
- **Inputs:** repo (GitHub URL, cloned to a temp dir with a size cap) and an issue URL or pasted text. The Groq key comes from `st.secrets` or a session-only password field. It is never logged or written to disk.
- **Run loop:** `graph.stream(..., thread_id=st.session_state.thread_id)`. Each node shows as a step in `st.status`, with the repro test, diffs and sandbox output in expanders.
- **HITL:** when the graph returns an interrupt, the UI renders the diff and test results with **Approve** and **Reject + feedback** buttons. The click calls `graph.invoke(Command(resume=...), config)`. `commit_and_pr` still runs only after this.
- **Commit/PR on the cloud:** the default is "download patch / commit message". Pushing a branch and opening a PR is opt-in and uses a user-supplied GitHub token held in session state only.
- **Persistence:** `MemorySaver` is per process, so state is lost on app restart or sleep. Use `PostgresSaver` (`CHECKPOINTER=postgres`) if runs must survive restarts.
- **Abuse and cost:** the 8-iteration cap, per-session run limit and token-aware pruning bound Groq usage. A public app should either ask users for their own key or be restricted. A shared key invites quota exhaustion.
- **Deploy:** the repo needs `requirements.txt` (or `pyproject.toml`) with `streamlit`, secrets set in the Community Cloud UI (`.streamlit/secrets.toml` is gitignored locally), and `SANDBOX_BACKEND=local` there. Free-tier apps have limited RAM and sleep when idle.
|