--- title: The app/ package kind: reference package: app --- # `app/` โ€” the Gradio web app The long-lived Gradio server that fronts the agent: the Hugging Face Space entrypoint that loads the embedding model once, streams a live reasoning trace while it works, and delegates every answer to `agent/`. ## Why this package exists / its boundary `app/` owns **UI and serving concerns only**. It paints the chat surface, keeps the server up, manages concurrency and rate limiting, streams progress, and formats the final markdown. It does **not** answer questions โ€” the moment a question needs answering, `_pipeline()` hands off to `agent/` (`agent.guard`, `agent.route.answer_routed`) and `index/` (`index.embed`, `index.freshness`). The split is deliberate: the batch GitHub Actions runs and the eval harness call the same `agent/` code with no Gradio in sight, and the app stays a thin, replaceable shell. If you are looking for retrieval, tool-calling, or JSON validation logic, it is not here โ€” it is in `agent/`. What lives here is everything that turns a validated `Answer` object into a responsive web page. ## The request lifecycle in the UI `respond()` (`app/main.py`) is a **generator**. Gradio streams every value it yields to the same output `Markdown` component, so the screen updates in place with no page refresh. The sequence per question: 1. **Thinking note** โ€” yields the static `THINKING_NOTE` ("๐Ÿ”Ž Searching the PyTorch docsโ€ฆ") immediately, so the first paint happens the instant Enter is pressed. 2. **Live grey trace + spinner** โ€” the real work (`_pipeline`) runs on a daemon **worker thread**. While it is alive, the generator loops every `THINKING_TICK` (0.6 s), reads the trace lines the pipeline has appended, and yields them via `_render_trace()`: each step in the theme's subdued grey with a turning Braille wheel (`THINKING_SPINNER`) on a trailing line. A multi- second wait reads as visible work rather than a freeze. 3. **Black answer** โ€” when the worker finishes, the generator yields the final markdown (`render(answer)`), replacing the whole grey trace with the answer in normal text. 4. **Freshness spinner** โ€” if the answer has citations and `index.freshness` is enabled, a stale-while-revalidate pass starts. The answer stays fully on screen while a **bare** wheel (just the spinner, no words) turns underneath it via `run_below()` โ€” the user reads while the app verifies the cited pages. 5. **(Maybe) regenerated answer** โ€” a clean check just drops the spinner. If the cited docs **drifted**, `freshness.refresh_pages()` self-heals the stored copies, `answer_routed` is re-run against the fresh content, and the new answer swaps in with the `โ†ป FRESHNESS_NOTE`. A failed check or failed regeneration silently keeps the answer already shown. ### Why stream the reasoning trace and not answer tokens The obvious move โ€” stream answer tokens like a chatbot โ€” does not fit this pipeline. The answer is **not free prose**; it is a validated JSON `Answer` object assembled over several tool calls (search โ†’ read โ†’ generate โ†’ static check). There are no partial tokens to emit until the whole thing is built and validated. So instead the app streams the **reasoning**: the pipeline emits a short trace line per step (which docs it searched, what it found, when it starts writing) through the `progress` sink, and `respond()` renders those live. The grey trace is honest visible progress for a request whose payload can only arrive all at once. ## Serving & robustness | Concern | Mechanism (in `app/main.py`) | Why | | --- | --- | --- | | **Warmup** | `_warm_up()` calls `index.embed.embed_query("warmup")` once at startup | Loads bge-small (~130 MB) before the first request, so no user eats the cold-start; also warms the guard's topicality embed. Best-effort โ€” a warmup failure is logged, not fatal. | | **Concurrency** | `demo.queue(default_concurrency_limit=CONCURRENCY)` | Gradio defaults every event to serial `concurrency_limit=1`; opening the queue lets many I/O-bound requests (LLM + Neon) overlap so nobody waits in line. `max_threads` is lifted in step (`max(40, CONCURRENCY*2)`) so the thread pool โ€” which also holds threads parked in 429 back-off โ€” never becomes the hidden ceiling. | | **Backpressure** | `max_size=QUEUE_SIZE` | Bounds how many requests may wait behind the workers. Under a flood, extra callers get "queue full" instead of an unbounded, forever-growing line. | | **Per-client rate limit** | `_rate_limited()` โ€” sliding window keyed on client IP | At most `RATE_LIMIT` questions per `RATE_WINDOW` seconds per IP, so one over-eager caller can't occupy every worker slot and burn the shared free-tier LLM quota. Set `RATE_LIMIT=0` to disable. The bucket table self-prunes past 4096 entries. | | **Fail-open errors** | `try/except` in `_pipeline` and in the worker `work()` | The UI must never crash or hang. Any exception is logged with type + message and the user gets the generic `ERROR_NOTE`; the real error never reaches the browser, since an exception string can leak hosts, model slugs, and config internals. | | **Smoke-test contract** | `ERROR_NOTE` contains the literal phrase **"went wrong"** | The post-deploy smoke test (`scripts/smoke_space.py`) greps for that marker to detect a broken Space. Keep the phrase. The `respond` event is also registered with `api_name="respond"` so the smoke test has a stable `client.predict(..., api_name="/respond")` endpoint; because `gradio_client.predict` returns the *last* yielded value, the generator's final yield is always a real answer. | ## UX decisions & rationale - **Send-on-Enter requires `lines=1`.** The question `Textbox` is `lines=1` (not 2), and this is load-bearing: Gradio only fires `Textbox.submit` on a **bare Enter for a single-line box**. A multi-line box (`lines>1`) treats Enter as a newline and submits on Shift+Enter instead. `max_lines=6` still lets a long question grow visually โ€” the submit rule keys off the `lines` prop, not the rendered height โ€” so Enter keeps sending. Do not bump `lines` back to 2. - **Theme-aware subdued grey that survives the sanitiser.** Trace lines use the inline style `color:var(--body-text-color-subdued)` (`TRACE_STYLE`), a Gradio CSS variable that adapts to light/dark theme. The inline style survives Gradio's markdown sanitiser (verified on gradio 6.20), which would strip a `