Hugging Face's logo

Spaces:
build-small-hackathon
/
agent-swarm-workbench
Paused

App Files Community
agent-swarm-workbench / ROADMAP.md
Kiy-K's picture
Kiy-K
Restore full Space app with bucket model support
31c8075 verified
preview code
|
Raw
History Blame Contribute Delete
9.12 kB

A newer version of the Gradio SDK is available: 6.22.0

Upgrade

ROADMAP

Goal

Build Agent Swarm Workbench: a prompt-first agent swarm workbench for Hugging Face Spaces.

The user provides one prompt for a codebase. The system creates a persisted Run, uses one DeepAgent with multiple subagents to plan and build the codebase, archives the result as a downloadable zip, restores that archive into a clean Validator sandbox, runs generated validation checks, and returns a summary with changed files and a Validation report.

Locked Product Decisions

  • The deliverable is a Codebase.
  • The user can download the Codebase as a .zip.
  • The MVP Swarm is one DeepAgent with multiple subagents.
  • The Coordinator generates validation checks from the prompt and user Criteria.
  • User-defined tests or Criteria are first-class inputs.
  • Validation is mandatory before a Run can be considered complete.
  • Validation runs in a separate Validator sandbox, never in the live worker Workspace.
  • The final output is Summary + changed files + Validation report + Codebase zip.
  • Keep the Python package name arena for now; the repo and public product name are agent-swarm-workbench / Agent Swarm Workbench.

Architecture Ground Rules

  • Keep FastAPI and Gradio thin over ArenaService.
  • Keep DeepAgent construction behind create_session().
  • Keep the Swarm runtime behind a small module interface.
  • Keep persistence behind small store interfaces.
  • Keep archive persistence behind the Codebase archive seam.
  • Keep validation orchestration in LangGraph.
  • Keep the worker Workspace and Validator sandbox separate.
  • Preserve a runnable app after every phase.
  • New product work must use Run, Swarm, Task, Codebase, Criteria, Validator, and Validation report language.
  • Legacy Match, Host, Participant, Challenge, Leaderboard, Judge, and Score language is quarantined outside the active app path.

Current Assets To Reuse

  • Python/FastAPI/Gradio app shell.
  • ArenaService application seam.
  • DeepAgents session factory and subagent support.
  • Local and Daytona sandbox adapters.
  • Upstash Redis REST client and store patterns.
  • Local and Redis-backed archive storage.
  • Sandboxed command executor used by the Validator.
  • Repo skill catalog mounted at /skills/.

Current Refactor Status

All nine phases are complete. The project is now Agent Swarm Workbench.

  • arena/run_flow.py owns the prompt-first Run lifecycle.
  • arena/codebase_archive.py is the only Codebase archive seam.
  • arena/validator_executor.py exposes a Validator-named sandbox adapter and factory.
  • arena/gradio_app.py is prompt-first only: prompt, Criteria, user tests, Run status, task table, event table, Validation report, and Codebase zip link.
  • arena/agent.py keeps DeepAgents as the Swarm SDK with subagents and tool interrupts.
  • arena/api.py and arena/service.py expose Run, skills, archive download, event streaming, and Validator operations.
  • arena/run_models.py and arena/validation_models.py provide prompt-first import surfaces; arena/models.py contains only active Run/Validator models.
  • arena/redis_client.py is the only source of RedisClient, UpstashRedisRestClient, and RedisStoreError.
  • Legacy match_store.py, match_flow.py, vibecoder.py, and judge_graph.py are deleted.
  • Package name arena kept.

MVP Success Criteria

  1. User starts a Run from one prompt and optional Criteria.
  2. Run receives a hidden rejoin token and survives refresh.
  3. One DeepAgent Swarm creates a task plan and works through subagents.
  4. Event stream shows plan, task progress, artifact creation, validation, and final summary.
  5. The Swarm produces a Codebase archive.
  6. The Codebase archive is downloadable as a zip.
  7. Validator graph restores the Codebase archive into a clean Validator sandbox.
  8. Validator runs generated checks and user-defined tests/Criteria-derived checks.
  9. Final response includes summary, changed files, Validation report, and risks.
  10. Redis-backed Run records and Codebase archives survive process restart.

Phase 1: Run Domain Models

Purpose: introduce the new domain language in code without broad renames.

Add models:

  • Run
  • RunRequest
  • RunView
  • RunStatus
  • RunTask
  • TaskStatus
  • RunArtifact
  • RunEvent
  • Criteria
  • ValidationCheck
  • ValidationCheckResult
  • ValidationReport
  • CodebaseArchive

Done when:

  • Tests validate Run creation, task state, event ordering, artifact pointers, Criteria, and Validation report shape.
  • Legacy match models still pass while migration continues.

Phase 2: Run Store

Purpose: persist prompt-first Runs behind a real seam.

Adapters:

  • InMemoryRunStore
  • RedisRunStore

Behavior:

  • Create Run from prompt and Criteria.
  • Issue Rejoin token.
  • Load public Run view without exposing secrets.
  • Rejoin by token.
  • Append Run events.
  • Save task state.
  • Save Codebase archive pointer.
  • Save Validation checks and Validation report.
  • Restore after process restart.
  • Mark active tasks interrupted if their runtime is gone.

Done when:

  • Fake Redis tests cover create, rejoin, event append, task update, artifact update, validation update, and restart recovery.

Phase 3: DeepAgent Swarm Runtime

Purpose: express the MVP Swarm as one DeepAgent with multiple subagents.

Subagents:

  • planner
  • coder
  • reviewer
  • test-runner
  • validator-prep

Behavior:

  • Seed User prompt and Criteria as read-only context.
  • Work inside /workspace.
  • Stream normalized events.
  • Record changed files.
  • Produce a Codebase archive through the archive seam.
  • Return a summary of work and known risks.

Done when:

  • One local fake Run can execute through the Swarm runtime and produce a Codebase archive.
  • Legacy runner concepts are replaced by SwarmRuntime.

Phase 4: Codebase Archive And Zip Download

Purpose: make the Codebase a durable, downloadable artifact.

Behavior:

  • Archive all files from /workspace.
  • Persist archive locally or in Redis depending on provider.
  • Generate a .zip from the archived Codebase.
  • Expose zip download through Gradio and FastAPI.
  • Keep file size caps and path validation.

Done when:

  • A Run produces a zip after completion.
  • Zip generation works after process restart from Redis archive.

Phase 5: Validator Graph

Purpose: retarget judge sandbox machinery into mandatory validation under Validator-named seams.

Files:

  • arena/validator_graph.py
  • arena/validator_executor.py
  • arena/codebase_handoff.py
  • arena/codebase_executor.py

Behavior:

  1. Restore Codebase archive into a clean Validator sandbox.
  2. Run all generated Validation checks.
  3. Run user-defined tests when provided.
  4. Optionally run LLM review over prompt, Criteria, changed files, and command outputs.
  5. Produce a Validation report.
  6. Attach the Validation report to the Run.

Status: done for the active app path.

Note: the old Judge executor alias has been removed. Validator restore now goes through the Codebase handoff module.

Done when:

  • Primary service path is validate_run().
  • Judge language is not exposed by FastAPI or Gradio.

Phase 6: Swarm Flow

Purpose: replace Match lifecycle with prompt-to-codebase Run orchestration.

Behavior:

  • Start Run.
  • Ask Coordinator to plan tasks.
  • Execute one DeepAgent Swarm runtime.
  • Persist events and task updates.
  • Archive Codebase.
  • Run Validator.
  • Mark Run completed only after mandatory validation.

Done when:

  • A Run can complete end-to-end in local fake mode.
  • Restart keeps archived Codebase and Validation report.
  • Active work lost after restart becomes interrupted.

Phase 7: Prompt-First Gradio UI

Purpose: remove competition controls from the user-facing app.

Status: done for the Hackathon demo shell. Legacy Match/Judge controls are no longer exposed in Gradio.

UI:

  • Prompt input.
  • Optional Criteria/tests input.
  • Start Run.
  • Rejoin token.
  • Event stream.
  • Task table.
  • Changed files.
  • Validation report.
  • Zip download.
  • Final summary.

Remove from primary UI:

  • Host tab.
  • Join tab.
  • Match code.
  • Participant controls.
  • Leaderboard.

Done when:

  • User can run the full workflow without touching internal IDs.

Phase 8: Run API

Purpose: expose prompt-first endpoints.

Primary endpoints (current):

  • POST /runs
  • GET /runs/{run_id}
  • POST /runs/rejoin
  • POST /runs/{run_id}/validate
  • GET /runs/{run_id}/codebase.zip

Streaming endpoint:

  • GET /runs/{run_id}/events for streamed Run events.

Done when:

  • FastAPI tests cover prompt-first Run lifecycle.
  • Legacy match endpoints are not exposed by the active FastAPI app.

Phase 9: Product Rename Cleanup

Purpose: delete or quarantine old arena language after behavior is migrated.

Status: done for the public product and repository name.

Done when:

  • UI title, README, ROADMAP, CONTEXT, AGENTS, and tests use swarm/run language.
  • Legacy match modules are deleted.
  • No active-path module under arena/ imports from arena/match_store.py, arena/match_flow.py, arena/vibecoder.py, or arena/judge_graph.py.
  • python scripts/task.py test passes after the cleanup.
  • Python package name arena kept.