--- title: Fabella emoji: ๐Ÿ“– colorFrom: green colorTo: yellow sdk: gradio sdk_version: 6.18.0 app_file: app.py pinned: true hf_oauth: true license: apache-2.0 short_description: Small words for big questions. datasets: - build-small-hackathon/fabella-traces tags: - track:backyard - sponsor:openbmb - sponsor:openai - sponsor:nvidia - sponsor:modal - achievement:offbrand - achievement:sharing - achievement:fieldnotes --- # Fabella **Small words for big questions.** Tell Fabella what's going on in a sentence or two. She drafts a short, kind, age-appropriate explanation you can read aloud โ€” a second small model checks it against a six-criterion rubric before you see it. **Submission for the [Build Small Hackathon](https://huggingface.co/spaces/build-small-hackathon/README) ยท Track I ยท Backyard AI.** [Live demo](https://build-small-hackathon-fabella.hf.space) ยท [Public GitHub repo](https://github.com/Kiy-K/Fabella) ยท [HF Space repo](https://huggingface.co/spaces/build-small-hackathon/Fabella) ยท [Modal app](https://modal.com/apps/khoitruong071510/main/deployed/fabella) ## Demo video [Watch on YouTube](https://youtu.be/dAoy1GRbEV8) The 90-second walkthrough shows the parent flow (situation โ†’ age โ†’ tone โ†’ validated draft โ†’ read aloud), the 3-model pipeline (Gemma 4 E4B drafter ยท Nemotron 3 Nano judge ยท VoxCPM2 read-aloud), the HF Bucket memory layer, and the anonymized trace dataset. Narration is ElevenLabs (`eleven_multilingual_v2`, voice `Roger`); caption timings are derived from a Whisper `small.en` pass over the synthesized audio. **Social post:** [X / Twitter](https://x.com/Kiy_K127/status/2066356328466202914?s=20) Source code: [`Kiy-K/Fabella`](https://github.com/Kiy-K/Fabella) --- ## The neighbor next door This is the person I built Fabella for: a parent I know, at 9 p.m., trying to explain to a 6-year-old that the family dog was not coming back. She had already had a hard day. She did not have the words she wanted, and she did not have the bandwidth to draft them. She needed a second pair of eyes that could read what she was about to say and tell her whether it would land. **Backyard AI** is exactly that brief: solve a real problem for someone you actually know. Fabella solves it for a parent in the moment they need help most โ€” translating a hard adult situation into language a small child can hear, then having a second model double-check the draft before a human reads it. --- ## What it does A parent types one or two sentences about the situation: a parent's hospitalization, a house move, a pet dying, a refusal to buy a phone. They pick the child's age, the child's name, and a tone (gentle, matter-of-fact, playful). The app drafts an explanation in the shape **Opener โ†’ Body โ†’ Closer โ†’ optional "if they ask more"**, then a second small model judges the draft against a rubric. The parent reads it, clicks **New version** if it isn't right, or clicks **Read aloud** for VoxCPM2 narration. The rubric the judge scores against (six checks, all hard-coded in `judge.py`): 1. All three primary sections (opener, body, closer) are present and non-empty 2. Body length is appropriate (1โ€“3 short paragraphs, not a wall of text) 3. Vocabulary matches the child's age 4. No moralizing, no lecturing, no "you should feel..." 5. No scary or violent content beyond what the situation requires 6. No invented facts โ€” only what the parent actually said The parent sees the validated draft, not a raw model output. If the judge rejects, the drafter gets one revision pass. If it still fails, the rule-based fallback runs in `agent.py` so the parent always gets *something* usable. --- ## The two-model pipeline | Layer | Model | Size | Runtime | Why this model | Why this execution | |---|---|---|---|---|---| | **Drafter** | `google/gemma-4-E4B-it` | 4B | Modal A10G ยท vLLM | Apache 2.0, fast on short empathetic text, native tool calling | **LangGraph ReAct** โ€” needs the state machine (draft โ†’ validate โ†’ revise โ†’ end) with tool calls and middleware-driven early exit | | **Judge** | `nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16` | 4B | Modal A10G ยท vLLM | Follows structured-output instructions reliably | **Pydantic v2** + one LLM call + one repair retry โ€” task is bounded, no agent loop needed | | **Read aloud** | `openbmb/VoxCPM2` | ~2B | Modal L4 ยท FastAPI | Apache 2.0, 48 kHz, voice-description control | Separate FastAPI server; only called when the user clicks **Read aloud** | | **Voice note** | `nvidia/nemotron-3.5-asr-streaming-0.6b` | 0.6B | Modal T4 ยท NeMo | Small multilingual streaming ASR with language prompts | Optional **Record** button: transcribes a short parent voice note into the textbox for review before drafting | The split is deliberate. The drafter needs agentic machinery (state machine, tool calls, conditional edges, jump-to-end). The judge doesn't โ€” its job is "receive rubric + draft, return a structured verdict." Pydantic gives disciplined output, type safety, and a one-shot repair retry. Two layers, two files, two execution models: `agent.py` for the loop, `judge.py` for the verdict. The core drafter/judge/read-aloud path uses **10B of parameters total**, with the largest single model at 4B. The optional voice-note input adds a separate 0.6B ASR model. The largest single model remains 4B, so Fabella is still a candidate for the **Tiny Titan** special award (โ‰ค4B). --- ## Sponsor prize notes - **OpenAI / Codex** โ€” Codex was used as a coding assistant for early boilerplate and scaffolding. This sponsor-track note is about development assistance, not runtime inference: Fabella's model pipeline uses Gemma, Nemotron, and VoxCPM2. - **NVIDIA** โ€” `nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16` is the second model in the pipeline and acts as the structured-output judge in `judge.py`. `nvidia/nemotron-3.5-asr-streaming-0.6b` powers the optional **Record** voice-note input. - **Modal** โ€” Modal runs the inference services: the Gemma drafter, the Nemotron judge, the VoxCPM2 TTS service, and the isolated T4 ASR experiment endpoint. - **OpenBMB** โ€” `openbmb/VoxCPM2` powers the optional **Read aloud** feature. --- ## Stack - **HF Space (CPU)** โ€” custom HTML + CSS + JS frontend served by `gradio.Server` (FastAPI subclass). Chat-style, parent-friendly UI: welcome screen with example situations, alternating parent / Fabella turns, per-turn Read-aloud button, no default Gradio chrome. - **HF OAuth** โ€” enabled for personalization; unsigned users fall back to browser-local anonymous sessions. - **HF Bucket per-user JSON** โ€” minimal chat history and parent preferences persist at `/data/fabella-data/user-.json` (signed-in users keyed by HF username, anonymous users keyed by a `localStorage` session ID). - **Voice note ASR** โ€” the optional **Record** button uses the browser `MediaRecorder`, sends a short base64 audio note to the Space's `transcribe_audio` API, and the Space proxies it to an isolated Modal T4 endpoint (`modal_asr_app.py`). The ASR endpoint follows NVIDIA NeMo's documented cache-aware streaming path (`set_inference_prompt`, `CacheAwareStreamingAudioBuffer`, `conformer_stream_step`) rather than plain `transcribe()`. - **Modal core app** โ€” one app, three web servers, all `min_containers=0` with a 2-minute `scaledown_window` so they cold-start on demand (3-day demo budget): - **Drafter** (A10G) โ€” vLLM with `--language-model-only --enable-auto-tool-choice --tool-call-parser gemma4 --enforce-eager --safetensors-load-strategy eager --max-model-len 8192` - **Judge** (A10G) โ€” vLLM with `--enforce-eager --safetensors-load-strategy eager --max-model-len 4096` (no tool-calling flags; Nemotron's tool-call dialect isn't a vLLM built-in) - **TTS** (L4) โ€” VoxCPM2 wrapped in a tiny FastAPI app on the smallest GPU that fits - **Image-baked weights**: drafter, judge, and TTS weights are baked into their respective images via `Image.run_function(download_*)`, so cold start is image-pull + eager-mode init + load-to-VRAM (no first-boot Volume read). - **Aggressive summarization in `agent.py`**: `_build_user_prompt` keeps the last 2 conversation turns verbatim and compresses everything older into a single short line capped at 320 chars. This is what lets us run the drafter at `--max-model-len 8192` instead of the model's nominal 32k, and it directly reduces per-request drafter token cost on long follow-up conversations. - **Cold-start tunings**: `--enforce-eager` skips CUDA-graph capture (saves 20โ€“40s of cold start at a small per-token throughput cost). `VLLM_DEEP_GEMM_WARMUP=skip` skips the dense-model MoE kernel warmup. `VLLM_USE_AOT_COMPILE=1` + `VLLM_CACHE_ROOT=/root/.cache/vllm` lets torch.compile artifacts persist across cold starts via the cache volume. - **No warmup ping on Space import.** The previous deployment fired a `/health` request to each endpoint on Space startup so the first parent click would land on a warm container. We removed it: every Space restart (code push, env-var change, periodic rebalance) paid for an A10G cold start whether or not a parent ever arrived. With the ping gone, the first request after a quiet period still pays a 30-60s cold start (image-baked weights, eager mode, AOT compile cache, deep-gemm warmup skip) and the 2-minute `scaledown_window` keeps a parent who reads the welcome screen and clicks a chip on a warm container for free. - **Modal ASR experiment app** โ€” separate `fabella-asr-experiment` deployment on T4 with `min_containers=0`; only wakes when a parent clicks **Record**. - **LangChain 1.x** ReAct loop with a custom middleware (`FabellaAgentMiddleware`) that jumps to `end` after a successful validation or after a hard cap of two tool calls. The `@hook_config(can_jump_to=["end"])` is required โ€” without it the early-exit silently does nothing. - **Pydantic v2** for the judge's structured output. `JudgeVerdict` has five fields (`ok`, `issues`, `score`, `verdict`, `reasoning`); cross-field consistency (`ok` โ‡” `verdict`) is enforced in code, not in the prompt. --- ## Output shape Every result is a four-section explanation in the parent's chosen tone: - **Opener** โ€” one sentence the parent can say to start the conversation - **Body** โ€” 1โ€“3 short paragraphs, second-person, age-appropriate, no moralizing - **Closer** โ€” one sentence to land the conversation - **If they ask another question** โ€” an optional follow-up the parent can use Example (situation: *"My 7-year-old's grandma is in the hospital for surgery. She keeps asking when grandma is coming home."*, tone: *gentle*): > **Opener:** I want to talk to you about Grandma. > **Body:** Grandma is in the hospital right now. She is having a little surgery. The doctors are taking good care of her. It is a part of getting her better. The doctors are very kind and they know just what to do. > **Closer:** We are all hoping she comes home very soon. > **If they ask more:** We can wait together and find out what the doctors say. --- ## Merit badges this submission is stacking Three claimed, three skipped. Fabella's honest inventory: | Badge | Status | Why | |---|---|---| | **Off-Brand** ๐ŸŽจ | Claimed | Custom HTML+CSS+JS frontend served by `gradio.Server` โ€” zero default Gradio chrome. | | **Sharing is Caring** ๐Ÿ“ก | Claimed | One anonymized row per request lands at [build-small-hackathon/fabella-traces](https://huggingface.co/datasets/build-small-hackathon/fabella-traces) โ€” schema, anonymization, and 5 seed rows in the public card. The Space publishes only when `FABELLA_SHARE_TRACES=1` is set (default `0`); parents can always pull their own data via the **Download my history** button regardless. | | **Field Notes** ๐Ÿ““ | Claimed | Blog/report on what was built and learned, by the maker. | | **Off the Grid** ๐Ÿ”Œ | Skipped | Drafter, judge, and TTS all run on Modal โ€” a cloud GPU platform, not "in front of you." | | **Well-Tuned** ๐ŸŽฏ | Skipped | No fine-tuning; Gemma 4 E4B-IT and Nemotron Nano 4B are used stock, no PEFT/LoRA, no published checkpoint on the Hub. | | **Llama Champion** ๐Ÿฆ™ | Skipped | Gemma 4 + Nemotron + VoxCPM2. No llama.cpp in the stack. | ## Off-Brand ๐ŸŽจ โ€” what to look for The hackathon's Off-Brand badge points at `gr.Server`. Fabella uses it. Concretely: - **`app.py:97`** โ€” `app = Server()` from `gradio`, not `gr.Blocks` or `gr.ChatInterface`. The Space has zero default Gradio chrome. - **`app.py:602`โ€“`1453`** โ€” `INDEX_HTML = r"""..."""`, ~850 lines of hand-written HTML, CSS, and vanilla JS. Custom welcome screen, example-situation chips, alternating parent/Fabella chat bubbles, per-turn Read-aloud buttons, settings dialog, history pane. - **`app.py:1454`** โ€” `@app.get("/", response_class=HTMLResponse) def index(): return INDEX_HTML` โ€” the only thing at `/` is the hand-coded page. - **No `gr.Blocks`, `gr.ChatInterface`, `gr.Tabs`, `gr.Interface`, or `with gr.` anywhere in `app.py`.** All UI state, all event handlers, and all the styling are in `INDEX_HTML` and the JS that lives inside it. - The demo video shows the running Space: the parent types a situation, the custom chat UI streams the four sections, and the per-turn Read-aloud button speaks the result. None of that ships with default Gradio. In other words: the canvas is `gradio.Server`'s FastAPI subclass, but the page is a hand-rolled SPA on top of it. Judges can verify by opening the Space, then running `grep -nE "gr\.Blocks|gr\.ChatInterface|gr\.Tabs" app.py` in the Space repo โ€” empty result. ## Agent trace data: parent self-export For this demo, the public dataset was removed by the maker. Parents pull their own data at any time via the **Settings โ†’ Download my history** button in the running Space, which calls `GET /api/history/download` and returns a JSON bundle: ```json { "schema": "fabella.history-bundle.v1", "exported_at": "2026-06-14T...", "owner_key": "anon:abcd1234...", "session_id": "abcd1234...", "signed_in": false, "profile": {"child_name": "Mira", "child_age": 7, "preferred_tone": "gentle"}, "messages": [{"role": "parent", "content": "...", "age": 7, "tone": "gentle", "created_at": "..."}, ...], "memory": {"facts": [...], "summary": "...", "threads": [...], "history_turns": 4}, "trace_publication": { "dataset": "build-small-hackathon/fabella-traces", "url": "https://huggingface.co/datasets/build-small-hackathon/fabella-traces", "this_session_max_published_rows": 3, "this_session_max_turns": 4, "anonymization": [ "Child name is dropped from the request and replaced with [name] in the draft.", "Raw situation text is never stored; only its SHA-256 hash, the first 60 chars, and its length are kept.", "Freeform history turns are replaced with role + length counts in the published row.", "The drafter's static system prompt is shipped in full (it's a public string in this repo)." ] } } ``` **Re-deploying the public dataset:** the `trace.py` publisher and the `build-small-hackathon/fabella-traces` schema are still in the repo. Set `FABELLA_SHARE_TRACES=1` on the Space to resume writing rows to that dataset. With the env var unset (or `0`), the publisher is a no-op and rows only live in the per-parent bucket. --- ## Files - `app.py` โ€” `gradio.Server` app, custom HTML+CSS+JS, `@app.api()` endpoint, HF OAuth-aware history APIs, no-op `@spaces.GPU` placeholder for HF runtime - `agent.py` โ€” LangChain ReAct drafter, `validate_explanation` tool, `FabellaAgentMiddleware` - `judge.py` โ€” Pydantic-validated judge with one repair retry, cross-field consistency enforcement - `schema.py` โ€” `ExplainRequest` dataclass + `JudgeVerdict` Pydantic model + `JudgeFailed` exception - `llm.py` โ€” `FabellaVLLM` BaseChatModel wrapping vLLM's OpenAI-compatible API; `bind_tools` builds the OpenAI-spec `tools=[...]` payload - `modal_app.py` โ€” Modal deployment (drafter + judge on A10G, VoxCPM2 TTS on L4) - `memory.py` โ€” bucket-backed parent memory and preference summaries for follow-up continuity - `safety.py` โ€” input sanitization, profanity block, `explain_to_words(tone)` - `trace.py` โ€” anonymized ReAct-trace capture and Hub publishing for the [fabella-traces](https://huggingface.co/datasets/build-small-hackathon/fabella-traces) dataset --- ## Run locally Local dev uses `uv`, not `pip`. The frontend runs on CPU; the three Modal inference containers must be live. ```bash uv venv .venv source .venv/bin/activate uv pip install -r requirements.txt export MODAL_DRAFTER_URL=https://khoitruong071510--fabella-serve-drafter.modal.run export MODAL_JUDGE_URL=https://khoitruong071510--fabella-serve-judge.modal.run export MODAL_TTS_URL=https://khoitruong071510--fabella-serve-tts.modal.run python app.py ``` Runtime notes: - `app.py` exposes `demo = app` for Gradio hot reload, while still launching the `gradio.Server` instance directly in normal runs. - The custom frontend calls `/gradio_api/call/make_explanation` with all nine API inputs, including `share_trace`, so Gradio's queue input validation matches the Python handler signature. - Known Hugging Face OAuth and Gradio/Starlette deprecation warnings are filtered at startup; they do not affect Space behavior. --- ## Constraints honored - **โ‰ค 32B params** ยท both LLMs are 4B; total inference is 10B - **Gradio app** ยท hosted as an HF Space, custom UI served by `gradio.Server` - **No API key needed for the models** ยท all open weights on Modal credits - **Show, don't tell** ยท [demo video](https://youtu.be/dAoy1GRbEV8) + [X social post](https://x.com/Kiy_K127/status/2066356328466202914?s=20)