File size: 19,337 Bytes
eb60226
26228ef
eb60226
26228ef
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
eb60226
26228ef
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
eb60226
26228ef
 
 
 
 
 
 
 
 
 
 
 
 
 
eb60226
26228ef
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
eb60226
26228ef
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
96b8c55
26228ef
 
 
 
 
 
 
 
 
 
 
 
96b8c55
26228ef
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
eb60226
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26228ef
 
 
 
 
 
 
 
96b8c55
26228ef
 
 
 
 
 
 
 
 
96b8c55
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
# Autonomous Bug-Fixing Agent — Architecture Spec

Status: **DRAFT v0.1 — for review.** No implementation exists yet; every file under `src/` is a stub.

## 1. Goal & Flow

Input: a GitHub issue / bug description + a local repo path.
Output: a reviewed, human-approved commit (and optionally a PR) containing a fix and a regression test.

```mermaid
flowchart TD
    A[ingest_issue] --> B[inspect_repo]
    B --> C[write_repro_test]
    C --> D[run_repro_test]
    D -->|fails as expected| E[propose_fix]
    D -->|passes: not reproduced, retries left| C
    D -->|passes: retries exhausted| X[escalate]
    E --> F[apply_fix]
    F --> G[run_tests]
    G -->|pass| H[human_approval -- interrupt]
    G -->|fail, iteration < 8| I[analyze_failure]
    G -->|fail, iteration >= 8| X
    I --> E
    H -->|approve| J[commit_and_pr]
    H -->|reject + feedback| I
    H -->|reject, abort| X
    J --> Z((END))
    X --> Z
```

Every LLM-calling node passes through the **context pruner** and the **Groq rate-limit wrapper** (section 6).

## 2. State Schema (`src/agent/state.py`)

The graph state is a `TypedDict` (LangGraph requirement for reducers). Structured sub-objects are **Pydantic v2** models, so they are validated at node boundaries and serialize cleanly through checkpointers (`model_dump(mode="json")`).

### 2.1 Pydantic models

| Model | Fields |
|---|---|
| `IssueInput` | `title: str`, `body: str`, `url: str \| None`, `number: int \| None`, `repo_path: Path` |
| `RepoContext` | `tree: str` (truncated listing), `relevant_files: dict[str, str]` (path → content snippet), `test_command: str`, `language: str` |
| `ReproTest` | `path: str`, `content: str`, `framework: Literal["pytest"]`, `confirmed_failing: bool` |
| `FilePatch` | `path: str`, `unified_diff: str` |
| `FixAttempt` | `iteration: int`, `rationale: str`, `patches: list[FilePatch]`, `result: SandboxResult \| None` |
| `SandboxResult` | `exit_code: int`, `stdout: str`, `stderr: str`, `timed_out: bool`, `duration_s: float`, `truncated: bool` (shared with the sandbox interface; defined in `docker_runner.py`, re-exported by `state.py`) |
| `ApprovalDecision` | `approved: bool`, `feedback: str \| None`, `reviewer: str \| None`, `decided_at: datetime` |
| `RunBudget` | `max_iterations: int = 8`, `max_repro_attempts: int = 2`, `max_context_tokens: int = 6000`, `max_tool_output_chars: int = 4000` |

### 2.2 `AgentStatus` (StrEnum)

`INGESTED`, `INSPECTED`, `REPRO_WRITTEN`, `REPRO_CONFIRMED`, `FIXING`, `TESTS_PASSING`, `AWAITING_APPROVAL`, `APPROVED`, `COMMITTED`, `ESCALATED`, `REJECTED`.

### 2.3 `AgentState` (TypedDict)

```python
class AgentState(TypedDict, total=False):
    issue: IssueInput
    repo: RepoContext
    repro_test: ReproTest
    attempts: Annotated[list[FixAttempt], merge_attempts] # history, upserted by iteration
    messages: Annotated[list[AnyMessage], add_messages]   # LLM conversation (pruned)
    iteration: int            # fix iterations consumed (0..max_iterations)
    repro_attempts: int       # repro-test generations consumed
    budget: RunBudget
    status: AgentStatus
    approval: ApprovalDecision | None
    branch: str | None
    commit_sha: str | None
    pr_url: str | None
    error: str | None         # terminal failure reason, set by escalate
```

Rules:
- Nodes return **partial updates only**; they never mutate the input state.
- `attempts` is full history for the audit trail. Its reducer `merge_attempts` upserts by `iteration`, so `run_tests` can return an updated copy of the latest attempt with its `SandboxResult` attached (a plain append reducer cannot update an entry). The LLM never sees it raw; the pruner builds a compact summary (section 6).
- `messages` is the only field the pruner rewrites.

## 3. Node Functions (`src/agent/nodes.py`)

All nodes: `(state: AgentState, config: RunnableConfig) -> dict` (partial update). Dependencies (LLM, sandbox, git tools) come from `config["configurable"]`, so tests can inject fakes.

| Node | Responsibility | Reads | Writes |
|---|---|---|---|
| `ingest_issue` | Validate/normalize the issue (parse a GitHub URL via API if given, else use the raw text); init budget and counters. | `issue` | `issue`, `budget`, `iteration=0`, `status` |
| `inspect_repo` | Build the repo tree, grep for symbols/stack traces named in the issue, read the top-N relevant files (size-capped), detect the test command. Uses `file_tools`. | `issue` | `repo`, `status` |
| `write_repro_test` | LLM writes a minimal failing pytest test; saved into the working tree. | `issue`, `repo`, `messages` | `repro_test`, `repro_attempts+1`, `messages` |
| `run_repro_test` | Run the repro test in the sandbox. We **expect failure**. A pass means the bug was not reproduced. | `repro_test` | `repro_test.confirmed_failing`, `status`, `messages` |
| `propose_fix` | LLM returns structured output (`FixProposal`: rationale + patches) via `with_structured_output`. | pruned context | `attempts` (+1, result=None), `messages` |
| `apply_fix` | Apply unified diffs in the working tree (`file_tools.apply_patch`); on patch failure, record the error and loop back without spending a sandbox run. | `attempts[-1]` | working tree, `attempts[-1]` error |
| `run_tests` | Run the repro test plus the existing suite in the sandbox; attach `SandboxResult` to the latest attempt; `iteration += 1`. | `repo.test_command` | `attempts[-1].result`, `iteration`, `status` |
| `analyze_failure` | LLM reads the (truncated) failing output + human feedback if any; produces diagnosis for the next `propose_fix`. | `attempts[-1]`, `approval` | `messages` |
| `human_approval` | Builds the review payload (diff, test result, rationale) and calls `langgraph.types.interrupt(payload)`. Resumed with `Command(resume=ApprovalDecision-dict)`. **The only path to `commit_and_pr`.** | `attempts`, `repro_test` | `approval`, `status` |
| `commit_and_pr` | `git_tools`: create branch `agent/fix-<issue#>`, stage, commit, optionally push + open PR via `gh`/API. Runs only on approved state. | `approval`, `issue` | `branch`, `commit_sha`, `pr_url`, `status=COMMITTED` |
| `escalate` | Terminal failure: write a report (what was tried, last output) and set `error`. Never commits. | all | `error`, `status=ESCALATED` |

## 4. Conditional Edge Routing (`src/agent/graph.py`)

Pure functions of state (no I/O), so they are trivially unit-testable.

```python
def route_after_repro(state) -> Literal["propose_fix", "write_repro_test", "escalate"]:
    if state["repro_test"].confirmed_failing:             return "propose_fix"
    if state["repro_attempts"] < state["budget"].max_repro_attempts: return "write_repro_test"
    return "escalate"

def route_after_apply(state) -> Literal["run_tests", "analyze_failure"]:
    return "analyze_failure" if state["attempts"][-1].patch_error else "run_tests"

def route_after_tests(state) -> Literal["human_approval", "analyze_failure", "escalate"]:
    if state["attempts"][-1].result.exit_code == 0:       return "human_approval"
    if state["iteration"] < state["budget"].max_iterations: return "analyze_failure"
    return "escalate"

def route_after_approval(state) -> Literal["commit_and_pr", "analyze_failure", "escalate"]:
    a = state["approval"]
    if a.approved:                                        return "commit_and_pr"
    if a.feedback and state["iteration"] < state["budget"].max_iterations:
        return "analyze_failure"                          # reviewer asked for changes
    return "escalate"
```

Invariants (each gets a test):
1. `iteration` never exceeds `max_iterations` (8). LangGraph `recursion_limit` is additionally set to a safe bound (about 4× nodes × iterations) as a hard backstop.
2. `commit_and_pr` has exactly one inbound edge: from `human_approval` via `route_after_approval`.
3. Timeouts and `SandboxResult.timed_out` count as failures, never as passes.

### Checkpointing & HITL
- `build_graph(checkpointer: BaseCheckpointSaver | None = None)` compiles with `MemorySaver()` by default.
- `get_checkpointer(settings)` factory: `MemorySaver` for dev; `PostgresSaver.from_conn_string(DATABASE_URL)` when `CHECKPOINTER=postgres` (extra dependency `postgres`; `.setup()` called once at startup).
- State must stay JSON-serializable for Postgres. This is why the models are Pydantic and `Path` is serialized as `str`.
- Runs are keyed by `thread_id`. A paused run is resumed with `graph.invoke(Command(resume=decision), config)`.

## 5. Docker Sandbox Interface (`src/sandbox/docker_runner.py`)

Library: `docker` (docker-py). The runner is a **long-lived container per agent run** (so installed state and edits persist across iterations). Commands run via `exec_run`.

```python
class SandboxConfig(BaseModel):
    image: str = "bugfix-agent-sandbox:latest"   # built from src/sandbox/Dockerfile
    workdir: str = "/workspace"
    mem_limit: str = "512m"
    nano_cpus: int = 1_000_000_000               # 1 CPU
    pids_limit: int = 256
    network_disabled: bool = True                # default-deny network
    default_timeout_s: int = 60
    max_output_bytes: int = 64_000               # stdout/stderr each, tail-preserved
    user: str = "sandbox"                        # non-root

class SandboxResult(BaseModel):
    exit_code: int; stdout: str; stderr: str
    timed_out: bool; duration_s: float; truncated: bool

class SandboxError(Exception): ...               # daemon unavailable, image missing, container died

class DockerSandbox:
    def __init__(self, config: SandboxConfig | None = None, client: DockerClient | None = None): ...
    def start(self, host_repo_path: Path) -> None   # create container, mount working copy
    def run(self, command: str, timeout_s: int | None = None) -> SandboxResult   # bash -c
    def run_tests(self, test_command: str, timeout_s: int | None = None) -> SandboxResult
    def write_file(self, path: str, content: str) -> None
    def read_file(self, path: str) -> str
    def stop(self) -> None                         # idempotent; force-remove container
    def __enter__(self) / __exit__(...)            # guarantees cleanup
    @staticmethod
    def is_available() -> bool                     # daemon reachable
```

Design decisions:
- **Isolation:** non-root user, `cap_drop=["ALL"]`, `security_opt=["no-new-privileges"]`, memory, CPU and PID limits, network off by default. Dependencies are pre-baked into the image. If a repo needs installs, there is an explicit opt-in `network_disabled=False` setup phase.
- **Working copy, not the original repo:** the agent works on a `git worktree` or temp copy of the repo and bind-mounts it at `/workspace`. The user's checkout is untouched until approval.
- **Timeouts:** enforced inside the container with `timeout <n> bash -c ...`. Exit code 124 maps to `timed_out=True`. This avoids orphaned processes, which a host-side timeout on `exec_run` would leave behind.
- **Output cap:** stdout/stderr are tail-truncated to `max_output_bytes` (failures appear at the end of pytest output). The state/prompt layer truncates again to about 4k chars for the LLM.
- **Cleanup:** the context manager and `stop()` always remove the container, including on exceptions.

## 6. Groq Free-Tier Guardrails

| Concern | Mechanism | Location |
|---|---|---|
| 429 / transient errors | `tenacity` `@retry(wait=wait_random_exponential(multiplier=2, max=60), stop=stop_after_attempt(6), retry=retry_if_exception_type(RateLimitError, APIConnectionError, ...))`. Honors the `retry-after` header when present. | `src/agent/llm.py` (added in a later phase) |
| Runaway loops | `max_iterations = 8` in `RunBudget` + routing invariants + LangGraph `recursion_limit`. | state, graph |
| Context growth | `prune_messages()` runs before every LLM call: (1) always keep system prompt, issue, and latest repro test; (2) replace old attempts with a one-line summary each (`iter 3: patched foo.py, test X failed: AssertionError`); (3) keep only the latest sandbox output, truncated to `max_tool_output_chars`; (4) final guard via `langchain_core.messages.trim_messages(max_tokens=max_context_tokens, strategy="last")`. | `src/agent/nodes.py` / helper |
| TPM bursts | Lightweight client-side pacing (min interval between calls) plus capped `max_tokens` on completions. | `llm.py` |
| Model | `ChatGroq(model="llama-3.3-70b-versatile", temperature=0)`, `GROQ_API_KEY` from env. | `llm.py` |

## 7. Project Layout

```
.
├── SPEC.md
├── pyproject.toml
├── .env.example
├── .gitignore
├── src/
│   ├── agent/
│   │   ├── state.py        # models + AgentState                (phase 1)
│   │   ├── graph.py        # build_graph, routers, checkpointer  (phase 3)
│   │   ├── nodes.py        # node functions + pruner              (phase 3)
│   │   └── prompts.py      # system / task prompts                (phase 3)
│   ├── sandbox/
│   │   ├── docker_runner.py   # DockerSandbox                     (phase 1)
│   │   └── Dockerfile         # python:3.12-slim + pytest, non-root user
│   └── tools/
│       ├── file_tools.py   # read/list/grep/apply_patch, path-jail  (phase 2)
│       └── git_tools.py    # branch/commit/push/PR                   (phase 2)
└── tests/
    ├── conftest.py              # `docker` marker, skip if daemon down
    ├── unit/
    │   ├── test_state.py            (phase 1)
    │   ├── test_docker_runner.py    (phase 1; integration-marked)
    │   └── test_routing.py          (phase 3)
    └── fixtures/
        └── buggy_repo/          # tiny repo with a known bug + expected repro test
```

## 8. Phases

1. **Phase 1:** `state.py`, `docker_runner.py`, Dockerfile, with unit tests that run real containers (marked `docker`, auto-skipped if there is no daemon).
2. **Phase 2:** `file_tools.py` (path-jailed to the repo root), `git_tools.py`.
3. **Phase 3:** `prompts.py`, `nodes.py`, `graph.py`, `llm.py`, routing tests with a fake LLM.
4. **Phase 4:** End-to-end run against `fixtures/buggy_repo`, CLI entry point, README.
5. **Phase 5:** Streamlit UI and deployment (section 11). Requires the sandbox backend abstraction (section 10), which is built right after Phase 2.

## 9. Open Questions for Review

1. **Language scope:** Python/pytest only for v1? (This spec assumes yes.)
2. **PR creation:** `gh` CLI vs. PyGithub / REST. Spec leans `gh` for fewer dependencies, with push + PR optional (commit-only by default).
3. **Repo isolation:** `git worktree` (requires a git repo) vs. plain temp-dir copy. Spec leans worktree.
4. **Network in the sandbox:** off by default with pre-baked deps. OK, or do you want an install phase with network enabled?
5. **LLM client:** `langchain-groq` (`ChatGroq`) is assumed over a raw OpenAI-compatible client.


## 10. Sandbox Backends (added for Streamlit deployment)

Streamlit Community Cloud has no Docker daemon, so the sandbox becomes pluggable.

- `src/sandbox/base.py`: a `Sandbox` `Protocol` with `start(path)`, `run`, `run_tests`, `write_file`, `read_file`, `stop`, and context-manager support. `SandboxResult` and `SandboxError` move here and are re-exported from `docker_runner.py`, so nothing else changes.
- `DockerSandbox` (existing): full isolation (section 5). The default for local use and any host with Docker.
- `LocalSandbox` (`src/sandbox/local_runner.py`): runs commands as subprocesses in the working-copy directory, with the same timeout and tail-truncated output contract.
  - **Weaker isolation, and the UI says so.** It does not block the network, jail the filesystem, or cap memory.
  - It scrubs the environment: `GROQ_API_KEY`, `GITHUB_TOKEN`, `DATABASE_URL`, and anything matching `*KEY*`, `*TOKEN*`, `*SECRET*`, so LLM-generated code cannot read secrets (Streamlit secrets are exposed as environment variables).
  - The directory jail is enforced in `write_file`/`read_file` only. Restrict it to demo-sized repos cloned into a temp directory.
- `create_sandbox(settings)` picks the backend from `SANDBOX_BACKEND=docker|local|auto`. `auto` uses Docker if the daemon is reachable, otherwise `LocalSandbox` with a logged warning. Nodes depend only on the `Sandbox` protocol.
- Side benefit: agent nodes and the graph can be tested end to end without Docker.

## 11. Streamlit App (Phase 5)

- **Entry point:** `src/ui/streamlit_app.py`; extra dependency group `ui = ["streamlit"]`.
- **Inputs:** repo (GitHub URL, cloned to a temp dir with a size cap) and an issue URL or pasted text. The Groq key comes from `st.secrets` or a session-only password field. It is never logged or written to disk.
- **Run loop:** `graph.stream(..., thread_id=st.session_state.thread_id)`. Each node shows as a step in `st.status`, with the repro test, diffs and sandbox output in expanders.
- **HITL:** when the graph returns an interrupt, the UI renders the diff and test results with **Approve** and **Reject + feedback** buttons. The click calls `graph.invoke(Command(resume=...), config)`. `commit_and_pr` still runs only after this.
- **Commit/PR on the cloud:** the default is "download patch / commit message". Pushing a branch and opening a PR is opt-in and uses a user-supplied GitHub token held in session state only.
- **Persistence:** `MemorySaver` is per process, so state is lost on app restart or sleep. Use `PostgresSaver` (`CHECKPOINTER=postgres`) if runs must survive restarts.
- **Abuse and cost:** the 8-iteration cap, per-session run limit and token-aware pruning bound Groq usage. A public app should either ask users for their own key or be restricted. A shared key invites quota exhaustion.
- **Deploy:** the repo needs `requirements.txt` (or `pyproject.toml`) with `streamlit`, secrets set in the Community Cloud UI (`.streamlit/secrets.toml` is gitignored locally), and `SANDBOX_BACKEND=local` there. Free-tier apps have limited RAM and sleep when idle.