Monike123's picture
Initial OpenEnv Agent Submission
d72844a
|
Raw
History Blame Contribute Delete
7.99 kB

Development Roadmap β€” Git Conflict Resolver

Format: Time-boxed phases (one-day sprint structure). Each phase has a clear goal, deliverables, and acceptance criteria. Log blockers immediately in docs/error_and_solving_chat.md.


Overview

Phase Time Block Focus Status
Phase 1 Morning Environment Core πŸ”² Not started
Phase 2 Late Morning Task Repositories πŸ”² Not started
Phase 3 Afternoon Agent + Reward πŸ”² Not started
Phase 4 Late Afternoon Evaluation Pipeline πŸ”² Not started
Phase 5 Evening Deployment πŸ”² Not started
Phase 6 End of Day Final Validation πŸ”² Not started

Morning Phase β€” Core Environment Build

Time target: Start of day β†’ noon Goal: A fully runnable environment with all 3 OpenEnv APIs working.

1. Environment API (env/conflict_env.py)

Build the OpenEnv-standard API surface:

reset()      β†’ state dict
step(action) β†’ (next_state, reward, done, info)
state()      β†’ state dict

State schema:

{
  "files": {"filename.py": "<content>"},
  "conflict_count": int,
  "syntax_valid": bool,
  "tests_passing": bool,
  "done": bool
}

Action schema:

{
  "file": "filename.py",
  "resolution": "<fully resolved file content as string>"
}

Acceptance criteria:

  • reset() returns a valid state dict with conflict_count > 0
  • step(action) returns a 4-tuple with correct types
  • state() returns the same schema without side effects
  • Entire flow runnable with python -c "from env.conflict_env import ConflictEnv; e = ConflictEnv('easy'); print(e.reset())"

2. Repository Sandbox Manager (env/repo_manager.py)

Handles isolation of task repos so each episode is fresh:

  • Copy task directory to a temp workspace on reset()
  • Apply file patches on step()
  • Clean up temp workspace on episode end or reset()

Key functions:

snapshot(task_path)          β†’ temp_path
apply_patch(temp_path, file, content)
cleanup(temp_path)

Acceptance criteria:

  • Two concurrent environments do not share file state
  • reset() always returns to original conflict state regardless of previous steps

Late Morning Phase β€” Task Repository Creation

Time target: 10:00 β†’ 12:30 Goal: 3 reproducible task repos with conflict markers and matching unit tests.

3. Task 1 β€” Easy Single-File Conflict (tasks/easy_conflict/)

Structure:

easy_conflict/
β”œβ”€β”€ calculator.py          ← has 1-2 conflict blocks
└── test_calculator.py     ← tests that fail while conflicts exist

Conflict example in calculator.py: ```python def add(a, b): <<<<<<< HEAD return a + b

return a + b + 1

feature/add-bonus


After correct resolution, `test_calculator.py` must pass.

---

### 4. Task 2 β€” Medium Multi-File Conflict (`tasks/medium_conflict/`)

Structure:

medium_conflict/ β”œβ”€β”€ utils.py ← signature changed in conflict (adds new param) β”œβ”€β”€ main.py ← calls utils with OLD signature = breaks after merge └── test_main.py ← tests that validate end-to-end call chain


The agent must:
1. Resolve conflict in `utils.py` (pick the new signature)
2. Update `main.py` call site to use new signature
3. Both files must be syntax-valid
4. Tests in `test_main.py` must pass

---

### 5. Task 3 β€” Hard Semantic Merge (`tasks/hard_conflict/`)

Structure:

hard_conflict/ β”œβ”€β”€ processor.py ← both branches added different logic to same function └── test_processor.py ← tests requiring BOTH branches' behavior


Naive resolution ("mine" or "theirs") will fail tests. The agent must combine both branches' additions intelligently.

---

## Afternoon Phase β€” Agent + Reward System

**Time target:** 12:30 β†’ 16:00
**Goal:** Working agent with all 5 skill modules and a calibrated reward engine.

### 6. Conflict Resolution Engine

**Modules:**
- `agent/conflict_detector.py` β€” parses conflict markers into structured blocks
- `agent/resolver.py` β€” decides how to resolve each block
- `agent/validator.py` β€” validates syntax after resolution

**Conflict detector internals:**

conflict_detector.detect_conflicts(content) β†’ [ConflictBlock]

ConflictBlock: .head_lines: list[str] .incoming_lines: list[str] .start_line: int .end_line: int .raw_block: str


**Resolver strategies (in order of preference):**
1. Rule-based: if one side is a subset of the other, pick the superset
2. AST-diff based: compare AST trees and merge non-overlapping nodes
3. LLM-assisted: call an LLM with the conflict block + context as prompt

---

### 7. Reward Function (`env/reward_engine.py`)

Dense reward β€” calculated at every `step()`:

| Component | Logic | Value |
|---|---|---|
| Marker removal | Count blocks removed this step | `+0.15 Γ— n_blocks_resolved` |
| Syntax valid (post-step) | `ast.parse()` of all files passes | `+0.20` |
| Syntax invalid (post-step) | Any file fails `ast.parse()` | `βˆ’0.10` |
| All tests pass | `pytest` exits with code 0 | `+0.30` |
| Semantic duplication | AST node similarity > threshold | `βˆ’0.20` |

**Max reward per episode:**

3 Γ— 0.15 + 0.20 + 0.30 = 0.95


**Reward function signature:**
```python
compute_reward(
    state_before: dict,
    state_after: dict,
    action: dict
) β†’ float

Late Afternoon Phase β€” Evaluation Pipeline

Time target: 16:00 β†’ 18:00 Goal: End-to-end evaluation loop that runs all 3 tasks and reports aggregate metrics.

8. Evaluation Runner (eval.py)

python eval.py --task easy
python eval.py --task medium
python eval.py --task hard
python eval.py --all

Output per task:

Task: easy_conflict
  Steps: 3
  Reward: 0.95 / 0.95  βœ…
  Conflict markers: 0 remaining
  Syntax: valid
  Tests: 3/3 passing

Task: medium_conflict
  Steps: 5
  Reward: 0.80 / 0.95
  ...

Acceptance criteria:

  • Easy task achieves reward >= 0.80 deterministically
  • Medium task achieves reward >= 0.60
  • Hard task agent makes progress (reward > 0.0)

Evening Phase β€” Deployment

Time target: 18:00 β†’ 21:00 Goal: Public Hugging Face Space with working Gradio demo.

9. Hugging Face Space (app.py)

UI components:

Component Description
Task Selector Dropdown: Easy / Medium / Hard conflict
Before Panel Shows original file content with conflict markers highlighted
After Panel Shows resolved file content
Diff View Unified diff between before and after
Reward Panel Per-component reward breakdown table
Run Button Triggers a full agent episode
Log Panel Step-by-step episode trace

Dependencies for Space:

gradio
gitpython
pytest

End of Day β€” Final Validation

Time target: 21:00 β†’ end Goal: All 3 tasks verified, deployed, demo publicly accessible.

10. Final Validation Checklist

  • Run pytest tests/ β€” all pass
  • Run python eval.py --all β€” metrics match expectations
  • Visit Space URL β€” demo loads without error
  • Click "Run Agent" on easy task β€” correct resolution shown
  • Click "Run Agent" on medium task β€” cross-file fix visible
  • Click "Run Agent" on hard task β€” semantic merge demonstrated
  • Share Space URL in README.md
  • Push all changes to version control
  • Update HISTORY.md with final version entry

Risk Register

Risk Likelihood Mitigation
Hard task takes too long for LLM Medium Fall back to rule-based resolver for demo
Pytest subprocess fails on HF Space Low Use unittest directly, avoid subprocess
Git sandbox isolation breaks Medium Use shutil.copytree not symlinks
Semantic duplication detector too slow Medium Skip for v1, add as stretch goal
HF Space build fails Low Test locally with gradio before push