Monike123's picture
Initial OpenEnv Agent Submission
d72844a
|
Raw
History Blame Contribute Delete
7.99 kB
# Development Roadmap β€” Git Conflict Resolver
> **Format:** Time-boxed phases (one-day sprint structure).
> Each phase has a clear goal, deliverables, and acceptance criteria.
> Log blockers immediately in `docs/error_and_solving_chat.md`.
---
## Overview
| Phase | Time Block | Focus | Status |
|---|---|---|---|
| Phase 1 | Morning | Environment Core | πŸ”² Not started |
| Phase 2 | Late Morning | Task Repositories | πŸ”² Not started |
| Phase 3 | Afternoon | Agent + Reward | πŸ”² Not started |
| Phase 4 | Late Afternoon | Evaluation Pipeline | πŸ”² Not started |
| Phase 5 | Evening | Deployment | πŸ”² Not started |
| Phase 6 | End of Day | Final Validation | πŸ”² Not started |
---
## Morning Phase β€” Core Environment Build
**Time target:** Start of day β†’ noon
**Goal:** A fully runnable environment with all 3 OpenEnv APIs working.
### 1. Environment API (`env/conflict_env.py`)
Build the OpenEnv-standard API surface:
```
reset() β†’ state dict
step(action) β†’ (next_state, reward, done, info)
state() β†’ state dict
```
**State schema:**
```python
{
"files": {"filename.py": "<content>"},
"conflict_count": int,
"syntax_valid": bool,
"tests_passing": bool,
"done": bool
}
```
**Action schema:**
```python
{
"file": "filename.py",
"resolution": "<fully resolved file content as string>"
}
```
**Acceptance criteria:**
- [ ] `reset()` returns a valid state dict with `conflict_count > 0`
- [ ] `step(action)` returns a 4-tuple with correct types
- [ ] `state()` returns the same schema without side effects
- [ ] Entire flow runnable with `python -c "from env.conflict_env import ConflictEnv; e = ConflictEnv('easy'); print(e.reset())"`
---
### 2. Repository Sandbox Manager (`env/repo_manager.py`)
Handles isolation of task repos so each episode is fresh:
- Copy task directory to a temp workspace on `reset()`
- Apply file patches on `step()`
- Clean up temp workspace on episode end or `reset()`
**Key functions:**
```python
snapshot(task_path) β†’ temp_path
apply_patch(temp_path, file, content)
cleanup(temp_path)
```
**Acceptance criteria:**
- [ ] Two concurrent environments do not share file state
- [ ] `reset()` always returns to original conflict state regardless of previous steps
---
## Late Morning Phase β€” Task Repository Creation
**Time target:** 10:00 β†’ 12:30
**Goal:** 3 reproducible task repos with conflict markers and matching unit tests.
### 3. Task 1 β€” Easy Single-File Conflict (`tasks/easy_conflict/`)
Structure:
```
easy_conflict/
β”œβ”€β”€ calculator.py ← has 1-2 conflict blocks
└── test_calculator.py ← tests that fail while conflicts exist
```
Conflict example in `calculator.py`:
```python
def add(a, b):
<<<<<<< HEAD
return a + b
=======
return a + b + 1
>>>>>>> feature/add-bonus
```
After correct resolution, `test_calculator.py` must pass.
---
### 4. Task 2 β€” Medium Multi-File Conflict (`tasks/medium_conflict/`)
Structure:
```
medium_conflict/
β”œβ”€β”€ utils.py ← signature changed in conflict (adds new param)
β”œβ”€β”€ main.py ← calls utils with OLD signature = breaks after merge
└── test_main.py ← tests that validate end-to-end call chain
```
The agent must:
1. Resolve conflict in `utils.py` (pick the new signature)
2. Update `main.py` call site to use new signature
3. Both files must be syntax-valid
4. Tests in `test_main.py` must pass
---
### 5. Task 3 β€” Hard Semantic Merge (`tasks/hard_conflict/`)
Structure:
```
hard_conflict/
β”œβ”€β”€ processor.py ← both branches added different logic to same function
└── test_processor.py ← tests requiring BOTH branches' behavior
```
Naive resolution ("mine" or "theirs") will fail tests. The agent must combine both branches' additions intelligently.
---
## Afternoon Phase β€” Agent + Reward System
**Time target:** 12:30 β†’ 16:00
**Goal:** Working agent with all 5 skill modules and a calibrated reward engine.
### 6. Conflict Resolution Engine
**Modules:**
- `agent/conflict_detector.py` β€” parses conflict markers into structured blocks
- `agent/resolver.py` β€” decides how to resolve each block
- `agent/validator.py` β€” validates syntax after resolution
**Conflict detector internals:**
```
conflict_detector.detect_conflicts(content) β†’ [ConflictBlock]
ConflictBlock:
.head_lines: list[str]
.incoming_lines: list[str]
.start_line: int
.end_line: int
.raw_block: str
```
**Resolver strategies (in order of preference):**
1. Rule-based: if one side is a subset of the other, pick the superset
2. AST-diff based: compare AST trees and merge non-overlapping nodes
3. LLM-assisted: call an LLM with the conflict block + context as prompt
---
### 7. Reward Function (`env/reward_engine.py`)
Dense reward β€” calculated at every `step()`:
| Component | Logic | Value |
|---|---|---|
| Marker removal | Count blocks removed this step | `+0.15 Γ— n_blocks_resolved` |
| Syntax valid (post-step) | `ast.parse()` of all files passes | `+0.20` |
| Syntax invalid (post-step) | Any file fails `ast.parse()` | `βˆ’0.10` |
| All tests pass | `pytest` exits with code 0 | `+0.30` |
| Semantic duplication | AST node similarity > threshold | `βˆ’0.20` |
**Max reward per episode:**
```
3 Γ— 0.15 + 0.20 + 0.30 = 0.95
```
**Reward function signature:**
```python
compute_reward(
state_before: dict,
state_after: dict,
action: dict
) β†’ float
```
---
## Late Afternoon Phase β€” Evaluation Pipeline
**Time target:** 16:00 β†’ 18:00
**Goal:** End-to-end evaluation loop that runs all 3 tasks and reports aggregate metrics.
### 8. Evaluation Runner (`eval.py`)
```
python eval.py --task easy
python eval.py --task medium
python eval.py --task hard
python eval.py --all
```
**Output per task:**
```
Task: easy_conflict
Steps: 3
Reward: 0.95 / 0.95 βœ…
Conflict markers: 0 remaining
Syntax: valid
Tests: 3/3 passing
Task: medium_conflict
Steps: 5
Reward: 0.80 / 0.95
...
```
**Acceptance criteria:**
- [ ] Easy task achieves `reward >= 0.80` deterministically
- [ ] Medium task achieves `reward >= 0.60`
- [ ] Hard task agent makes progress (reward > 0.0)
---
## Evening Phase β€” Deployment
**Time target:** 18:00 β†’ 21:00
**Goal:** Public Hugging Face Space with working Gradio demo.
### 9. Hugging Face Space (`app.py`)
**UI components:**
| Component | Description |
|---|---|
| Task Selector | Dropdown: Easy / Medium / Hard conflict |
| Before Panel | Shows original file content with conflict markers highlighted |
| After Panel | Shows resolved file content |
| Diff View | Unified diff between before and after |
| Reward Panel | Per-component reward breakdown table |
| Run Button | Triggers a full agent episode |
| Log Panel | Step-by-step episode trace |
**Dependencies for Space:**
```
gradio
gitpython
pytest
```
---
## End of Day β€” Final Validation
**Time target:** 21:00 β†’ end
**Goal:** All 3 tasks verified, deployed, demo publicly accessible.
### 10. Final Validation Checklist
- [ ] Run `pytest tests/` β€” all pass
- [ ] Run `python eval.py --all` β€” metrics match expectations
- [ ] Visit Space URL β€” demo loads without error
- [ ] Click "Run Agent" on easy task β€” correct resolution shown
- [ ] Click "Run Agent" on medium task β€” cross-file fix visible
- [ ] Click "Run Agent" on hard task β€” semantic merge demonstrated
- [ ] Share Space URL in `README.md`
- [ ] Push all changes to version control
- [ ] Update `HISTORY.md` with final version entry
---
## Risk Register
| Risk | Likelihood | Mitigation |
|---|---|---|
| Hard task takes too long for LLM | Medium | Fall back to rule-based resolver for demo |
| Pytest subprocess fails on HF Space | Low | Use `unittest` directly, avoid subprocess |
| Git sandbox isolation breaks | Medium | Use `shutil.copytree` not symlinks |
| Semantic duplication detector too slow | Medium | Skip for v1, add as stretch goal |
| HF Space build fails | Low | Test locally with `gradio` before push |