| # ROADMAP |
|
|
| ## Goal |
|
|
| Build Agent Swarm Workbench: a prompt-first agent swarm workbench for Hugging Face Spaces. |
|
|
| The user provides one prompt for a codebase. The system creates a persisted Run, uses one DeepAgent with multiple subagents to plan and build the codebase, archives the result as a downloadable zip, restores that archive into a clean Validator sandbox, runs generated validation checks, and returns a summary with changed files and a Validation report. |
|
|
| ## Locked Product Decisions |
|
|
| - The deliverable is a **Codebase**. |
| - The user can download the Codebase as a `.zip`. |
| - The MVP Swarm is **one DeepAgent with multiple subagents**. |
| - The Coordinator generates validation checks from the prompt and user Criteria. |
| - User-defined tests or Criteria are first-class inputs. |
| - Validation is mandatory before a Run can be considered complete. |
| - Validation runs in a separate **Validator sandbox**, never in the live worker Workspace. |
| - The final output is Summary + changed files + Validation report + Codebase zip. |
| - Keep the Python package name `arena` for now; the repo and public product name are `agent-swarm-workbench` / Agent Swarm Workbench. |
|
|
| ## Architecture Ground Rules |
|
|
| - Keep FastAPI and Gradio thin over `ArenaService`. |
| - Keep DeepAgent construction behind `create_session()`. |
| - Keep the Swarm runtime behind a small module interface. |
| - Keep persistence behind small store interfaces. |
| - Keep archive persistence behind the Codebase archive seam. |
| - Keep validation orchestration in LangGraph. |
| - Keep the worker Workspace and Validator sandbox separate. |
| - Preserve a runnable app after every phase. |
| - New product work must use Run, Swarm, Task, Codebase, Criteria, Validator, and Validation report language. |
| - Legacy Match, Host, Participant, Challenge, Leaderboard, Judge, and Score language is quarantined outside the active app path. |
|
|
| ## Current Assets To Reuse |
|
|
| - Python/FastAPI/Gradio app shell. |
| - `ArenaService` application seam. |
| - DeepAgents session factory and subagent support. |
| - Local and Daytona sandbox adapters. |
| - Upstash Redis REST client and store patterns. |
| - Local and Redis-backed archive storage. |
| - Sandboxed command executor used by the Validator. |
| - Repo skill catalog mounted at `/skills/`. |
|
|
| ## Current Refactor Status |
|
|
| All nine phases are complete. The project is now Agent Swarm Workbench. |
|
|
| - `arena/run_flow.py` owns the prompt-first Run lifecycle. |
| - `arena/codebase_archive.py` is the only Codebase archive seam. |
| - `arena/validator_executor.py` exposes a Validator-named sandbox adapter and factory. |
| - `arena/gradio_app.py` is prompt-first only: prompt, Criteria, user tests, Run status, task table, event table, Validation report, and Codebase zip link. |
| - `arena/agent.py` keeps DeepAgents as the Swarm SDK with subagents and tool interrupts. |
| - `arena/api.py` and `arena/service.py` expose Run, skills, archive download, event streaming, and Validator operations. |
| - `arena/run_models.py` and `arena/validation_models.py` provide prompt-first import surfaces; `arena/models.py` contains only active Run/Validator models. |
| - `arena/redis_client.py` is the only source of `RedisClient`, `UpstashRedisRestClient`, and `RedisStoreError`. |
| - Legacy `match_store.py`, `match_flow.py`, `vibecoder.py`, and `judge_graph.py` are deleted. |
| - Package name `arena` kept. |
|
|
| ## MVP Success Criteria |
|
|
| 1. User starts a Run from one prompt and optional Criteria. |
| 2. Run receives a hidden rejoin token and survives refresh. |
| 3. One DeepAgent Swarm creates a task plan and works through subagents. |
| 4. Event stream shows plan, task progress, artifact creation, validation, and final summary. |
| 5. The Swarm produces a Codebase archive. |
| 6. The Codebase archive is downloadable as a zip. |
| 7. Validator graph restores the Codebase archive into a clean Validator sandbox. |
| 8. Validator runs generated checks and user-defined tests/Criteria-derived checks. |
| 9. Final response includes summary, changed files, Validation report, and risks. |
| 10. Redis-backed Run records and Codebase archives survive process restart. |
|
|
| ## Phase 1: Run Domain Models |
|
|
| Purpose: introduce the new domain language in code without broad renames. |
|
|
| Add models: |
|
|
| - `Run` |
| - `RunRequest` |
| - `RunView` |
| - `RunStatus` |
| - `RunTask` |
| - `TaskStatus` |
| - `RunArtifact` |
| - `RunEvent` |
| - `Criteria` |
| - `ValidationCheck` |
| - `ValidationCheckResult` |
| - `ValidationReport` |
| - `CodebaseArchive` |
|
|
| Done when: |
|
|
| - Tests validate Run creation, task state, event ordering, artifact pointers, Criteria, and Validation report shape. |
| - Legacy match models still pass while migration continues. |
|
|
| ## Phase 2: Run Store |
|
|
| Purpose: persist prompt-first Runs behind a real seam. |
|
|
| Adapters: |
|
|
| - `InMemoryRunStore` |
| - `RedisRunStore` |
|
|
| Behavior: |
|
|
| - Create Run from prompt and Criteria. |
| - Issue Rejoin token. |
| - Load public Run view without exposing secrets. |
| - Rejoin by token. |
| - Append Run events. |
| - Save task state. |
| - Save Codebase archive pointer. |
| - Save Validation checks and Validation report. |
| - Restore after process restart. |
| - Mark active tasks interrupted if their runtime is gone. |
|
|
| Done when: |
|
|
| - Fake Redis tests cover create, rejoin, event append, task update, artifact update, validation update, and restart recovery. |
|
|
| ## Phase 3: DeepAgent Swarm Runtime |
|
|
| Purpose: express the MVP Swarm as one DeepAgent with multiple subagents. |
|
|
| Subagents: |
|
|
| - planner |
| - coder |
| - reviewer |
| - test-runner |
| - validator-prep |
|
|
| Behavior: |
|
|
| - Seed User prompt and Criteria as read-only context. |
| - Work inside `/workspace`. |
| - Stream normalized events. |
| - Record changed files. |
| - Produce a Codebase archive through the archive seam. |
| - Return a summary of work and known risks. |
|
|
| Done when: |
|
|
| - One local fake Run can execute through the Swarm runtime and produce a Codebase archive. |
| - Legacy runner concepts are replaced by `SwarmRuntime`. |
|
|
| ## Phase 4: Codebase Archive And Zip Download |
|
|
| Purpose: make the Codebase a durable, downloadable artifact. |
|
|
| Behavior: |
|
|
| - Archive all files from `/workspace`. |
| - Persist archive locally or in Redis depending on provider. |
| - Generate a `.zip` from the archived Codebase. |
| - Expose zip download through Gradio and FastAPI. |
| - Keep file size caps and path validation. |
|
|
| Done when: |
|
|
| - A Run produces a zip after completion. |
| - Zip generation works after process restart from Redis archive. |
|
|
| ## Phase 5: Validator Graph |
|
|
| Purpose: retarget judge sandbox machinery into mandatory validation under Validator-named seams. |
|
|
| Files: |
|
|
| - `arena/validator_graph.py` |
| - `arena/validator_executor.py` |
| - `arena/codebase_handoff.py` |
| - `arena/codebase_executor.py` |
|
|
| Behavior: |
|
|
| 1. Restore Codebase archive into a clean Validator sandbox. |
| 2. Run all generated Validation checks. |
| 3. Run user-defined tests when provided. |
| 4. Optionally run LLM review over prompt, Criteria, changed files, and command outputs. |
| 5. Produce a Validation report. |
| 6. Attach the Validation report to the Run. |
|
|
| Status: done for the active app path. |
|
|
| Note: the old Judge executor alias has been removed. Validator restore now goes through the Codebase handoff module. |
|
|
| Done when: |
|
|
| - Primary service path is `validate_run()`. |
| - Judge language is not exposed by FastAPI or Gradio. |
|
|
| ## Phase 6: Swarm Flow |
|
|
| Purpose: replace Match lifecycle with prompt-to-codebase Run orchestration. |
|
|
| Behavior: |
|
|
| - Start Run. |
| - Ask Coordinator to plan tasks. |
| - Execute one DeepAgent Swarm runtime. |
| - Persist events and task updates. |
| - Archive Codebase. |
| - Run Validator. |
| - Mark Run completed only after mandatory validation. |
|
|
| Done when: |
|
|
| - A Run can complete end-to-end in local fake mode. |
| - Restart keeps archived Codebase and Validation report. |
| - Active work lost after restart becomes interrupted. |
|
|
| ## Phase 7: Prompt-First Gradio UI |
|
|
| Purpose: remove competition controls from the user-facing app. |
|
|
| Status: done for the Hackathon demo shell. Legacy Match/Judge controls are no longer exposed in Gradio. |
|
|
| UI: |
|
|
| - Prompt input. |
| - Optional Criteria/tests input. |
| - Start Run. |
| - Rejoin token. |
| - Event stream. |
| - Task table. |
| - Changed files. |
| - Validation report. |
| - Zip download. |
| - Final summary. |
|
|
| Remove from primary UI: |
|
|
| - Host tab. |
| - Join tab. |
| - Match code. |
| - Participant controls. |
| - Leaderboard. |
|
|
| Done when: |
|
|
| - User can run the full workflow without touching internal IDs. |
|
|
| ## Phase 8: Run API |
|
|
| Purpose: expose prompt-first endpoints. |
|
|
| Primary endpoints (current): |
|
|
| - `POST /runs` |
| - `GET /runs/{run_id}` |
| - `POST /runs/rejoin` |
| - `POST /runs/{run_id}/validate` |
| - `GET /runs/{run_id}/codebase.zip` |
|
|
| Streaming endpoint: |
|
|
| - `GET /runs/{run_id}/events` for streamed Run events. |
|
|
| Done when: |
|
|
| - FastAPI tests cover prompt-first Run lifecycle. |
| - Legacy match endpoints are not exposed by the active FastAPI app. |
|
|
| ## Phase 9: Product Rename Cleanup |
|
|
| Purpose: delete or quarantine old arena language after behavior is migrated. |
|
|
| Status: done for the public product and repository name. |
|
|
| Done when: |
|
|
| - UI title, README, ROADMAP, CONTEXT, AGENTS, and tests use swarm/run language. |
| - Legacy match modules are deleted. |
| - No active-path module under `arena/` imports from `arena/match_store.py`, `arena/match_flow.py`, `arena/vibecoder.py`, or `arena/judge_graph.py`. |
| - `python scripts/task.py test` passes after the cleanup. |
| - Python package name `arena` kept. |
|
|
