docs/idea_and_development_chat.md

Idea and Development Chat

Purpose: Log all architecture decisions, design explorations, open questions, and planning discussions here. Treat this as a searchable reasoning journal — write down WHY you made a decision, not just what it is. Format entries with a date and topic heading.

---

How to Use This File

When you are:

• Designing a new module → write the options you considered and why you chose one
• Changing an existing design → write what changed and what triggered the change
• Unsure about an approach → write the open question and leave it for later resolution
• Discussing with a collaborator or AI → paste key excerpts and conclusions

---

[2026-04-07] — Project Concept Selection

Problem Statement

Git merge conflicts are one of the most universally painful experiences in software development. When two branches touch the same lines:

• Git inserts conflict markers (<<<<<<<, =======, >>>>>>>) into the file
• The developer must manually inspect both versions and decide how to combine them
• In large codebases with hundreds of files, this is painstaking and error-prone
• No existing tool can autonomously resolve conflicts with high accuracy

Why This is a Good AI Benchmark Task

1. Structured input: Conflict markers provide exact boundaries of disagreement
2. Verifiable output: AST parsing and unit tests give objective success/fail signal
3. Multi-skill: Requires reading, reasoning, editing, and validation all in one episode
4. Scalable difficulty: Easy (trivial merge) → Hard (semantic reasoning required)
5. Real-world relevance: Every developer team faces this problem

Platform Decision: OpenEnv

OpenEnv was selected over custom environments because:

• Standard reset() / step() / state() API is familiar to RL researchers
• Compatible with multiple agent frameworks (RL policies, LLM tool-use, hybrid)
• Easy to wrap into a Gymnasium-compatible environment later
• Deployment on Hugging Face is straightforward

Alternative Considered: Pure LLM Benchmark

A simpler alternative would be to just prompt an LLM with a conflict and check the output. This was rejected because:

• No sandboxed execution — can't verify syntax or tests
• No episode structure — can't study multi-step resolution
• Can't support RL training loops
• Doesn't fit the "agent operating in environment" paradigm

---

[2026-04-07] — Task Difficulty Design

Design Question

How many difficulty levels, and what differentiates them?

Decision: 3 Levels

Level	Why It Exists
Easy	Baseline — any agent should pass this. Confirms environment works.
Medium	Tests cross-file reasoning. More realistic real-world scenario.
Hard	Tests semantic understanding. Requires genuine comprehension, not just marker removal.

Easy Task Design Rationale

A single Python file with 1–2 conflict blocks where:

• HEAD block adds simple logic
• Incoming block also adds simple logic
• The correct merge is to include BOTH additions (not pick one)

Or: HEAD and incoming change a return value differently — tests define which is correct.

The unit tests are the ground truth. The agent wins if tests pass.

Medium Task Design Rationale

Two files: utils.py and main.py.

utils.py gets a new function parameter in the incoming branch. The conflict sits in utils.py. After the agent resolves it by accepting the new signature, main.py now breaks — because it still calls the old signature.

The agent must:

1. Notice the signature changed
2. Find all call sites (in main.py)
3. Update them to match

This tests cross-file awareness — something LLMs frequently fail at in practice.

Hard Task Design Rationale

Both HEAD and incoming branches independently added useful logic to the same function body:

• HEAD: added input validation
• Incoming: added a new computation mode

A naive "take mine" or "take theirs" strategy fails the tests, because the tests verify BOTH behaviors. The agent must understand what each branch was trying to do and merge them.

This tests semantic reasoning — the hardest and most valuable capability.

---

[2026-04-07] — Reward Function Design

Design Question

Should the reward be sparse (only on success) or dense (every step)?

Decision: Dense Reward

Reasons:

1. Sparse reward (only +1 on full success) makes learning very hard — especially on hard tasks
2. Dense reward gives signal even for partial progress (removed 1 of 3 blocks)
3. Mixed reward (dense during episode + bonus on success) is the ideal approach for RL

Reward Component Breakdown

Why +0.15 for marker removal?

• Conflict removal is a necessary but not sufficient condition for success
• A lower value than syntax/test rewards because removing markers is easy (just delete them) without actually fixing the code
• The syntax and test rewards ensure the removal was valid

Why −0.10 for syntax error?

• The agent must be penalized for making the file worse
• A small negative prevents the agent from making random edits
• Not too large so that the agent can recover from mistakes

Why +0.30 for tests passing?

• This is the hardest to achieve and the most meaningful signal — worth the most
• Tests passing = end goal achieved (from a developer's perspective)

Why −0.20 for semantic duplication?

• Duplicated logic is a silent bug — tests may still pass but the code quality degrades
• This teaches the agent to produce clean merges, not just technically correct ones

Max Episode Reward Calculation

For a 3-block easy task:

3 blocks × 0.15 = 0.45   (all markers removed)
+ 0.20                   (syntax valid)
+ 0.30                   (tests pass)
= 0.95 max reward

For a multi-step episode, rewards accumulate across steps. The agent can also receive negative rewards if it introduces syntax errors along the way.

---

[2026-04-07] — Agent Architecture Decisions

Question: Rule-based vs LLM-based resolver?

Option A — Pure Rule-Based

• Pros: fast, deterministic, no API calls
• Cons: fails on semantic tasks, brittle to edge cases
• Valid for: easy task only

Option B — Pure LLM

• Pros: handles semantic reasoning, flexible
• Cons: slow, costly, non-deterministic
• Valid for: hard task

Decision: Hybrid (rule-first, LLM fallback)

if conflict is trivially resolvable (one side is subset of other):
    use rule-based resolver
elif conflict can be resolved by AST merge:
    use AST-based resolver
else:
    use LLM resolver with task context as prompt

This gives performance on easy tasks and capability on hard tasks.

Question: How does the agent receive state?

The agent receives a state dict from the environment:

{
  "files": {"main.py": "<content>"},
  "conflict_count": 3,
  "syntax_valid": False,
  "tests_passing": False,
  "done": False
}

The agent's action is:

{
  "file": "main.py",
  "resolution": "<full resolved content>"
}

One action = one file resolved. Multi-file tasks require multiple steps.

---

[2026-04-07] — Deployment Architecture

Hugging Face Space Design

User visits Space URL
  └─► Selects task (Easy / Medium / Hard)
  └─► Clicks "Run Agent"
       └─► Backend: env.reset() called
       └─► Agent runs episode
       └─► Results shown: before/after diff + reward table

Why Gradio?

• Built into HF Spaces natively
• Fast to prototype interactive demos
• supports code display, diffs, tables natively

Single-file app: app.py (everything in one file for Space simplicity)

---

Open Questions (To Be Resolved)

Question	Status
Should hard task use GPT-4 or a local LLM?	🔲 Open
Should multi-file actions be batched or sequential?	🔲 Open
Should the environment support Java/JS files, or Python only?	🔲 Open — Python only for v1
Should episode length be capped? (max N steps)	🔲 Open — suggest cap at 10 steps
Should we persist episode logs to disk for analysis?	🔲 Open