README.md

---
title: Nova
emoji: 🎙️
colorFrom: indigo
colorTo: purple
sdk: docker
app_port: 7860
pinned: false
short_description: Talk to an on-device voice agent on a Raspberry Pi
---

# nova-connect

Talk to **Nova** — a voice assistant whose speech recognition, speech synthesis and
wake word run on a Raspberry Pi 5 with a Hailo NPU — from any browser, anywhere.

This Space is only the front desk. It serves the page and signs a short-lived
LiveKit ticket after a passcode check. It never touches audio and never talks to
the Pi: the browser and the Pi each dial out to a LiveKit server and meet in a room.

### What runs where

```mermaid
flowchart TB
    subgraph B["Browser — laptop or phone, anywhere"]
        PAGE["Nova page<br/>orb · state · transcript · LLM switch · chime"]
    end

    subgraph HF["This Space — Hugging Face, Docker"]
        DESK["POST /api/session<br/>passcode → scrypt check → signed ticket"]
    end

    subgraph SFU["LiveKit server — self-hosted SFU"]
        ROOM["room nova-xxxx<br/>relays encrypted audio + data · no AI"]
    end

    subgraph PI["Raspberry Pi 5 + Hailo NPU — one nova-hailo process"]
        direction TB
        WORKER["LiveKit Agents worker<br/>registered as nova-&lt;hostname&gt;"]
        WAKE["wake word on the caller's audio<br/>openWakeWord · nova_pro_v3"]
        SESS["AgentSession<br/>Silero VAD · endpointing · interruptions"]
        STT["Parakeet STT (CPU)"]
        PIPE["NovaPipeline<br/>router · tools · search · research<br/>LLM: Groq ⇄ Qwen2 on Hailo · Piper TTS"]
        LAN["/v1/realtime — LAN door<br/>in-car Qt HMI"]
        LEASE["SessionLease — one conversation at a time"]
        WORKER --> WAKE --> SESS --> STT --> PIPE
        LAN --> PIPE
        LEASE -.- WORKER
        LEASE -.- LAN
    end

    PAGE -- "① passcode" --> DESK
    DESK -- "② url + ticket (10 min)" --> PAGE
    PAGE <-- "③ WebRTC audio + data" --> ROOM
    WORKER -. "outbound WSS, always on<br/>(no inbound port on the Pi)" .-> ROOM
    ROOM -- "④ dispatch to nova-&lt;hostname&gt;" --> WORKER
    PIPE -- "⑤ Nova's voice (PCM → Opus)" --> ROOM
```

### Wake word and follow-up

Same contract as the in-car HMI: nothing is transcribed while idle.

```mermaid
stateDiagram-v2
    [*] --> Idle: Nova joins the room
    Idle --> Listening: "Hey Nova" or tap the orb · chime
    Listening --> Thinking: you stop talking
    Thinking --> Speaking: first audio
    Speaking --> Listening: answer done · follow-up window restarts
    Speaking --> Listening: you interrupt
    Listening --> Idle: 5 s of silence (livekit.followup_s)
```

### One conversation, step by step

```mermaid
sequenceDiagram
    autonumber
    actor U as You (browser)
    participant S as This Space
    participant R as LiveKit SFU
    participant W as Pi worker
    participant N as NovaPipeline

    Note over W: at boot: registers as nova-<hostname>
    U->>S: POST /api/session {passcode}
    S-->>U: url + ticket (1 room, dispatch nova-<hostname>)
    U->>R: join room, publish mic
    R->>W: dispatch job
    W->>W: take SessionLease, or reply "Nova is in use"
    W->>R: join as agent
    W-->>U: nova.state idle
    U->>R: "Hey Nova"
    R->>W: mic audio (16 kHz copy to the wake word)
    W-->>U: nova.state listening → chime
    U->>R: "what's the weather in Chennai?"
    W->>W: VAD + endpointing → Parakeet → text
    W->>N: run_text_turn(text)
    N-->>W: Piper PCM, clause by clause
    W->>R: Nova's voice + transcript
    R-->>U: Nova speaks
    Note over U,W: 5 s of silence → nova.state idle
```

## Setting up a Pi

On the Pi, in `edge/nova-hailo`:

1. Install the extra: `uv sync --extra livekit` (plus any extras already in use).
2. Add the LiveKit server to `.env`: `LIVEKIT_URL`, `LIVEKIT_API_KEY`, `LIVEKIT_API_SECRET`.
3. Turn the online door on in the active profile config:
   ```yaml
   livekit:
     enabled: true
   ```
4. Start Nova as usual (`./scripts/run_demo_oem.sh`). The log prints the Pi's
   agent name, taken from its hostname:
   `LiveKit: registered as nova-<hostname>`.

The LAN realtime API keeps working unchanged; one conversation at a time across
both (a second caller hears "Nova is in use").

## Setting up this Space

1. Make a passcode: `python scripts/make_passcode.py` (shows the passcode once and
   prints its hash).
2. Space → Settings → **Secrets**:

   | Secret | Value |
   |---|---|
   | `LIVEKIT_URL` | the LiveKit server, `wss://…` |
   | `LIVEKIT_API_KEY` / `LIVEKIT_API_SECRET` | its key pair |
   | `NOVA_AGENT_NAME` | exactly the name the Pi printed, e.g. `nova-conmod-integration1` |
   | `NOVA_PASSCODE_HASH` | the `scrypt$…` line from step 1 |

3. `GET /api/health` shows `{"configured": true, ...}` once all five are set.

## Run locally

```bash
pip install -r requirements.txt
export LIVEKIT_URL=... LIVEKIT_API_KEY=... LIVEKIT_API_SECRET=... \
       NOVA_AGENT_NAME=nova-<hostname> NOVA_PASSCODE_HASH='scrypt$...'
python app.py          # http://localhost:7860
```

Tests: `python -m pytest tests -q`.

## Security notes

- The LiveKit API secret lives only in Space secrets and on the Pi.
- Tickets last 10 minutes, join one fresh room, may publish only the microphone
  and data, and dispatch only the configured Pi.
- Passcodes are checked against an scrypt hash; 5 attempts per 5 minutes per IP.
- Media is encrypted in transit (DTLS-SRTP); the LiveKit server can see it.