Spaces:
Running
Running
| title: Nova | |
| emoji: 🎙️ | |
| colorFrom: indigo | |
| colorTo: purple | |
| sdk: docker | |
| app_port: 7860 | |
| pinned: false | |
| short_description: Talk to an on-device voice agent on a Raspberry Pi | |
| # nova-connect | |
| Talk to **Nova** — a voice assistant whose speech recognition, speech synthesis and | |
| wake word run on a Raspberry Pi 5 with a Hailo NPU — from any browser, anywhere. | |
| This Space is only the front desk. It serves the page and signs a short-lived | |
| LiveKit ticket after a passcode check. It never touches audio and never talks to | |
| the Pi: the browser and the Pi each dial out to a LiveKit server and meet in a room. | |
| ### What runs where | |
| ```mermaid | |
| flowchart TB | |
| subgraph B["Browser — laptop or phone, anywhere"] | |
| PAGE["Nova page<br/>orb · state · transcript · LLM switch · chime"] | |
| end | |
| subgraph HF["This Space — Hugging Face, Docker"] | |
| DESK["POST /api/session<br/>passcode → scrypt check → signed ticket"] | |
| end | |
| subgraph SFU["LiveKit server — self-hosted SFU"] | |
| ROOM["room nova-xxxx<br/>relays encrypted audio + data · no AI"] | |
| end | |
| subgraph PI["Raspberry Pi 5 + Hailo NPU — one nova-hailo process"] | |
| direction TB | |
| WORKER["LiveKit Agents worker<br/>registered as nova-<hostname>"] | |
| WAKE["wake word on the caller's audio<br/>openWakeWord · nova_pro_v3"] | |
| SESS["AgentSession<br/>Silero VAD · endpointing · interruptions"] | |
| STT["Parakeet STT (CPU)"] | |
| PIPE["NovaPipeline<br/>router · tools · search · research<br/>LLM: Groq ⇄ Qwen2 on Hailo · Piper TTS"] | |
| LAN["/v1/realtime — LAN door<br/>in-car Qt HMI"] | |
| LEASE["SessionLease — one conversation at a time"] | |
| WORKER --> WAKE --> SESS --> STT --> PIPE | |
| LAN --> PIPE | |
| LEASE -.- WORKER | |
| LEASE -.- LAN | |
| end | |
| PAGE -- "① passcode" --> DESK | |
| DESK -- "② url + ticket (10 min)" --> PAGE | |
| PAGE <-- "③ WebRTC audio + data" --> ROOM | |
| WORKER -. "outbound WSS, always on<br/>(no inbound port on the Pi)" .-> ROOM | |
| ROOM -- "④ dispatch to nova-<hostname>" --> WORKER | |
| PIPE -- "⑤ Nova's voice (PCM → Opus)" --> ROOM | |
| ``` | |
| ### Wake word and follow-up | |
| Same contract as the in-car HMI: nothing is transcribed while idle. | |
| ```mermaid | |
| stateDiagram-v2 | |
| [*] --> Idle: Nova joins the room | |
| Idle --> Listening: "Hey Nova" or tap the orb · chime | |
| Listening --> Thinking: you stop talking | |
| Thinking --> Speaking: first audio | |
| Speaking --> Listening: answer done · follow-up window restarts | |
| Speaking --> Listening: you interrupt | |
| Listening --> Idle: 5 s of silence (livekit.followup_s) | |
| ``` | |
| ### One conversation, step by step | |
| ```mermaid | |
| sequenceDiagram | |
| autonumber | |
| actor U as You (browser) | |
| participant S as This Space | |
| participant R as LiveKit SFU | |
| participant W as Pi worker | |
| participant N as NovaPipeline | |
| Note over W: at boot: registers as nova-<hostname> | |
| U->>S: POST /api/session {passcode} | |
| S-->>U: url + ticket (1 room, dispatch nova-<hostname>) | |
| U->>R: join room, publish mic | |
| R->>W: dispatch job | |
| W->>W: take SessionLease, or reply "Nova is in use" | |
| W->>R: join as agent | |
| W-->>U: nova.state idle | |
| U->>R: "Hey Nova" | |
| R->>W: mic audio (16 kHz copy to the wake word) | |
| W-->>U: nova.state listening → chime | |
| U->>R: "what's the weather in Chennai?" | |
| W->>W: VAD + endpointing → Parakeet → text | |
| W->>N: run_text_turn(text) | |
| N-->>W: Piper PCM, clause by clause | |
| W->>R: Nova's voice + transcript | |
| R-->>U: Nova speaks | |
| Note over U,W: 5 s of silence → nova.state idle | |
| ``` | |
| ## Setting up a Pi | |
| On the Pi, in `edge/nova-hailo`: | |
| 1. Install the extra: `uv sync --extra livekit` (plus any extras already in use). | |
| 2. Add the LiveKit server to `.env`: `LIVEKIT_URL`, `LIVEKIT_API_KEY`, `LIVEKIT_API_SECRET`. | |
| 3. Turn the online door on in the active profile config: | |
| ```yaml | |
| livekit: | |
| enabled: true | |
| ``` | |
| 4. Start Nova as usual (`./scripts/run_demo_oem.sh`). The log prints the Pi's | |
| agent name, taken from its hostname: | |
| `LiveKit: registered as nova-<hostname>`. | |
| The LAN realtime API keeps working unchanged; one conversation at a time across | |
| both (a second caller hears "Nova is in use"). | |
| ## Setting up this Space | |
| 1. Make a passcode: `python scripts/make_passcode.py` (shows the passcode once and | |
| prints its hash). | |
| 2. Space → Settings → **Secrets**: | |
| | Secret | Value | | |
| |---|---| | |
| | `LIVEKIT_URL` | the LiveKit server, `wss://…` | | |
| | `LIVEKIT_API_KEY` / `LIVEKIT_API_SECRET` | its key pair | | |
| | `NOVA_AGENT_NAME` | exactly the name the Pi printed, e.g. `nova-conmod-integration1` | | |
| | `NOVA_PASSCODE_HASH` | the `scrypt$…` line from step 1 | | |
| 3. `GET /api/health` shows `{"configured": true, ...}` once all five are set. | |
| ## Run locally | |
| ```bash | |
| pip install -r requirements.txt | |
| export LIVEKIT_URL=... LIVEKIT_API_KEY=... LIVEKIT_API_SECRET=... \ | |
| NOVA_AGENT_NAME=nova-<hostname> NOVA_PASSCODE_HASH='scrypt$...' | |
| python app.py # http://localhost:7860 | |
| ``` | |
| Tests: `python -m pytest tests -q`. | |
| ## Security notes | |
| - The LiveKit API secret lives only in Space secrets and on the Pi. | |
| - Tickets last 10 minutes, join one fresh room, may publish only the microphone | |
| and data, and dispatch only the configured Pi. | |
| - Passcodes are checked against an scrypt hash; 5 attempts per 5 minutes per IP. | |
| - Media is encrypted in transit (DTLS-SRTP); the LiveKit server can see it. | |