README.md

metadata

title: Multi-Agent Assistant
emoji: 🤖
colorFrom: purple
colorTo: blue
sdk: gradio
app_file: app.py
pinned: false
sdk_version: 6.26.0

Multi-Agent Assistant

A chat app where a manager agent routes each request to specialist agents, built with LangGraph and running on Hugging Face ZeroGPU.

How it works

1. You send a message in the chat box.
2. A manager agent reads it and decides what's needed next: web search, Python execution, numeric analysis, or "I have enough — finish."
3. The chosen specialist agent does its job and reports back to the manager, which can loop through multiple specialists before finishing (capped at MAX_TURNS round-trips to avoid infinite loops).
4. A finalize step synthesizes everything the specialists found into one answer, which appears in the chat.

Gradio UI → Manager agent → {search_agent, code_agent, data_agent} → Manager → Finalize → UI

Specialists

Agent	Tool	What it does
search_agent	Tavily web search	Looks up current information on the web
code_agent	Restricted Python subprocess	Writes and runs Python to compute things
data_agent	Summary statistics	Computes count/sum/mean/min/max on a list of numbers

The model itself is Qwen/Qwen2.5-3B-Instruct, loaded locally and run on a ZeroGPU-allocated GPU per request (see generate() in agents.py) — nothing is sent to an external inference API.

Files

• app.py — Gradio UI (chat interface, session state)
• agents.py — LangGraph state machine, model loading, tools, prompts
• requirements.txt — Python dependencies (torch/gradio/spaces versions are managed by the Space's build system and intentionally not pinned here)

Required secrets

Set these under Settings → Variables and secrets:

Secret	Used for
TAVILY_API_KEY	Required for search_agent. Get a free key at tavily.com.
HF_TOKEN	Optional. Speeds up model downloads and avoids Hub rate limits.

Without TAVILY_API_KEY, every other feature still works — search_agent just returns a "Search unavailable" message instead of crashing the app.

Hardware

This Space requires ZeroGPU hardware (not CPU-basic), since agents.py runs local model inference inside a function decorated with @spaces.GPU.

Technical architecture

State (AgentState, LangGraph)

A TypedDict threaded through every node: messages (accumulated via add_messages), next (the manager's routing decision), task (the original request), results (a scratch dict specialists write into), and turn_count (see MAX_TURNS below).

Control flow

sequenceDiagram
    participant U as User
    participant M as Manager
    participant S as Specialist
    U->>M: task
    loop up to MAX_TURNS
        M->>M: generate() → route decision
        alt route = specialist
            M->>S: task
            S->>S: generate() → tool call or final
            S-->>M: result written to state.results
        else route = finish
            M->>M: break loop
        end
    end
    M->>M: finalize (synthesize results)
    M-->>U: answer

MAX_TURNS (default 6, in agents.py) hard-caps manager↔specialist round-trips in code — not just via LangGraph's recursion_limit, which is a blunter backstop covering every node hop including finalize. Once turn_count exceeds MAX_TURNS, manager_node skips the model call entirely and force-routes to finalize, so a flip-flopping manager can't burn extra inference calls on its way out.

Model layer (local inference on ZeroGPU)

• _model/_tokenizer are loaded once at import time, on CPU.
• generate(messages, max_new_tokens) is the single inference entry point used by every node (manager, all three specialists, finalize). It's decorated with @spaces.GPU(duration=60): ZeroGPU attaches a GPU only for the duration of this call, then releases it — no dedicated GPU sits idle between requests.
• Inside generate(): model moved to cuda, prompt built via apply_chat_template(..., return_dict=True) (the explicit dict form, to avoid ambiguity between transformers versions that return a raw tensor vs. a BatchEncoding — this was the root cause of an earlier AttributeError that only reproduced on this Space's installed version), greedy decoding (do_sample=False).
• All exceptions are caught inside generate() and returned as a {"error": ...} string rather than left to propagate across the @spaces.GPU process boundary — exceptions crossing that boundary raw can arrive mangled in the caller (observed in practice: a real AttributeError surfaced to the caller as an opaque KeyError('AttributeError')). Catching locally preserves the real exception type/message and logs the full traceback.

Manual tool-calling protocol

Local open-weight models don't expose a native function-calling API the way hosted inference endpoints do, so tool use is hand-rolled:

1. Each specialist's system prompt lists its tool's name, description, and argument schema.
2. The model is asked to respond with only JSON — either {"tool": "...", "args": {...}} or {"final": "..."}.
3. The node parses that JSON, invokes the named tool directly if present, then makes one follow-up generate() call asking the model to summarize the tool's raw output.
4. Malformed/non-JSON responses fall back to using the model's raw text as the summary, rather than failing the turn.

Configuration surface

Constant (agents.py)	Default	Effect
MODEL_ID	Qwen/Qwen2.5-3B-Instruct	Swap for a larger model if quality is the bottleneck
MAX_TURNS	6	Manager↔specialist round-trip cap
@spaces.GPU(duration=60)	60s	Per-call GPU allocation window — bump if generation runs long

Known limitations

• run_python executes in a locked-down subprocess (restricted builtins, 5s timeout, isolated interpreter mode) — a reasonable starting point, but not a real security sandbox. For untrusted input in production, use a dedicated sandbox such as E2B Code Interpreter or Modal Sandboxes instead.
• Specialists use a manually-parsed JSON tool-call protocol rather than a native function-calling API, since local open models don't expose one the way hosted inference endpoints do. Malformed JSON from the model falls back to using its raw text response.
• MAX_TURNS (in agents.py) hard-caps manager↔specialist round-trips per request to prevent runaway routing loops.
• ZeroGPU cold starts. The first request after the Space idles and restarts pays model-load latency. If generation routinely runs long (bigger model, longer max_new_tokens), duration=60 may need increasing.
• Single shared model instance. _model/_tokenizer are process-wide globals; concurrent requests from different users serialize through the same weights (ZeroGPU allocates the GPU per call, not per user).