Keys Ghost Machine — audio/visual agent, powered by TensorFold
The voices (Drama 3), the promo and screenshots for Boo, and previews of its optional lite voice (Kokoro-82M).
Fourteen named voices for Fish Audio Drama 3, made for Boo, the audio/visual agent in drowzeys/keys-ghost-machine-audio-visual-agent. Vesper is Boo's default.
What's new in v0.3.0
Boo moves to TensorFold 1.1.5 (the voices here are unchanged). Bolt on a DGX Spark runs on the stock 1.1.5 release;
Bolt and Cosmos3-Edge on the Mac, and Cosmos3-Edge on CUDA, run on the keys 1.1.5 builds. To update Boo, run the install
command again, then kgm restart; in Docker, pull image 0.3.0.
- Same answers, same speed: on an M5 Ultra Bolt's greedy replies and Cosmos3-Edge's camera answers are identical to the 1.1.4 builds (7 of 7 each) at the same speed (Bolt first token 77–81 ms, Cosmos3-Edge 78 ms).
- Drama 3 on the Mac stays on the keys 1.1.4-keys.1 engine (shipped beside 1.1.5 in the Mac archive). Stock 1.1.5 reads the NVFP4 pruned checkpoint on Metal and reuses a voice's reference prefill, but its first audio in a named voice was 107–343 ms against a steady 67–79 ms for the keys engine, so these voices keep speaking from the faster one. The DGX Spark's Drama 3 is unchanged too, and the v0.2.2 fixes (no keep-warm line on the Spark, cuBLASLt 13.1 in Docker) stay.
v0.2.2
Boo's Full voice (Drama 3) on a DGX Spark answers about twice as fast: 0.47–0.55 s voice to voice instead of
1.0–1.1 s (the voices here are unchanged). To update Boo, run the install command again, then kgm restart; in
Docker, pull image 0.2.2.
- No Drama 3 keep-warm line on the DGX Spark: its CUDA Drama 3 runs one synthesis at a time, so the throwaway "Mm." Boo spoke while you talked made the reply wait. The Spark profile now turns it off (Bolt stays warm).
- Docker: the TensorFold engines get the cuBLASLt 13.1 they are qualified with, so Drama 3's fast one-row kernel is on again.
v0.2.1
A fix release for v0.2.0 (superseded by v0.2.2).
- Fresh DGX Spark installs: the page could not connect after Parakeet warmed up. NeMo's CUDA-graph decoder wrote a
NUL into Python's shared
b" "object (an empty NVRTC log read intob" " * 1), which broke uvicorn's status lines and websockets' parser and left damaged.pycfiles. Boo now gives NVRTC its own buffer (the Docker image was not affected). - "(Boo says)" at the start of a reply: after a switch of voice engine (Full/Lite) or voice, the sample line "Hi, this is how I sound now." went into the conversation as "(Boo says) Hi, this is how I sound now.", and Bolt could copy the label, or the whole line, into its next replies. The sample line is no longer kept in the conversation, Boo's own short lines are kept without a label, and a reply that still starts with a stage direction such as "(Boo says)" loses it before it is spoken or stored.
- Docs: the Mac's Drama 3 first audio is ~60 ms on TensorFold 1.1.4 with the engine warm, as in a conversation (it said 122 ms, the figure after the engine has sat idle).
New in v0.2.0: Kokoro Lite, a second voice for smaller machines
Kokoro-82M (hexgrad, Apache-2.0) can now speak for Boo instead of Drama 3, or beside it with a switch in the page. It is for memory-constrained hardware: a DGX Spark that shares its memory with other work, or a Mac at the low end of the requirements. Everything else (Bolt, Cosmos3-Edge, Parakeet, Eyes of Kai, Hermes) stays the same.
- Install Lite, DGX Spark (about 48 GB free memory):
curl -fsSL https://raw.githubusercontent.com/drowzeys/keys-ghost-machine-audio-visual-agent/main/install.sh | bash -s -- --voice lite - Install Lite, Mac (Apple silicon, macOS 26+; 96 GB comfortable, 64 GB tight): the same command,
curl -fsSL https://raw.githubusercontent.com/drowzeys/keys-ghost-machine-audio-visual-agent/main/install.sh | bash -s -- --voice lite - Docker, Lite (DGX Spark, no Drama 3 mount needed):
docker run --gpus all --rm -p 8095:8095 -e VOICE_ENGINE=lite -e KAI=1 \ -v ~/keys-ghost-machine/models/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB:/models/bolt \ -v ~/keys-ghost-machine/models/cosmos3-tf:/models/cosmos3 -v ~/models/hf-cache:/models/hf-cache \ ghcr.io/drowzeys/keys-ghost-machine-audio-visual-agent:0.3.0 - Both, with a Full/Lite toggle: install with
--voice both(or-e VOICE_ENGINE=bothin Docker, with the Drama 3 mount) and the page shows a Full / Lite switch per conversation; the voice list follows the engine. On an existing install,kgm voice full|lite|bothswitches. Pick a Kokoro voice in the page's Voice list (bf_emmaby default,af_heartis Kokoro's best rated) or setKOKORO_VOICE=af_heartin Docker. - Trade-off: on a DGX Spark Lite answers in about 0.33–0.40 s voice to voice (Full 0.55–0.77 s) and uses about 40 GB less memory. On a Mac it is about as fast as Full and saves about 9 GB. Lite speaks English only in Boo and has no voice cloning, emotion control or the 14 Drama 3 voices.
Kokoro Lite vs Drama 3 (4-bit)
| Kokoro Lite | Drama 3 4-bit (NVFP4) | |
|---|---|---|
| Model | 82M parameters, ~330 MB | Billions of parameters, 4-bit NVFP4 |
| Voice-engine memory: DGX Spark | ~3.5 GB | ~42 GB |
| Voice-engine memory: Mac | ~0.8 GB | ~10 GB |
| Voice starts: Spark | 33–40 ms | 112–121 ms |
| Voice starts: Mac | 51–54 ms | ~60 ms |
| Voice to voice: Spark | 0.33–0.40 s | 0.55–0.77 s |
| Voice to voice: Mac | 0.49–0.64 s | 0.43–0.61 s |
| Naturalness (UTMOS) | 4.35 (bf_emma) – 4.52 (af_heart) | 4.53 (Vesper) |
| Voices | 54 built in | 14 custom, incl. Vesper |
| Voice cloning | No | Yes |
| Emotion/style control | No | Yes |
| Long-line stalls | None | Occasional; fixed by the silence guard + resume in v0.2.0 |
| Streaming | Whole sentence (<0.13 s) | Streams within a sentence |
| Languages | EN, ES, FR, IT, PT, HI, JA, ZH | Wider, incl. German |
| Licence | Apache-2.0 | Fish Audio Community License (hosted commercial use needs a Fish licence) |
Spark: Lite is twice as fast and frees ~40 GB. Mac: same speed, so run both and keep Vesper with a Lite toggle.
Choose Drama for a signature voice, cloning, emotion or German; choose Kokoro for small or shared hardware, an Apache
licence or the lowest delay.
Fixes in v0.2.0
- Drama 3 long-line stalls: a long reply that went quiet after its first sentence (on about 1 in 5 paragraph-length lines) is now caught. The DGX Spark's Drama 3 build gains TensorFold 1.1.4's silence guard (new in the v0.2.0 Linux archive and the Docker image), the Mac build already had it, and Boo itself drops a stalled take's quiet tail and asks for the unspoken rest.
- Never deaf after a fast short reply: a quick reply such as "I'm on it." could leave Boo marked as still speaking, and the session stopped listening. Fixed (64 of 64 turns heard in testing).
- Drama 3 "speech output capacity is full" (503): the keep-warm loop no longer outlives its page or a stopped
microphone (it stops on disconnect, after 2 s without audio and after 60 s), warms with one "Mm." at a time and never
while anything is being said; a refused line is asked again for up to
retry_s(6 s) instead of being lost; a stopped turn frees its speech slot at once. - Bolt repetition loops: a reply that starts repeating a sentence it already said ends there, unsaid.
- Interrupted replies stopped before their first word no longer leave "(interrupted)" in the history for the brain to copy.
- The reference Mac now runs
voice_engine: both, with the Full/Lite toggle tested live.
Builds: v0.3.0 release ·
docker pull ghcr.io/drowzeys/keys-ghost-machine-audio-visual-agent:0.3.0
Requirements
Drama 3 served by TensorFold with named voices (--voices): about 21 GB of weights (the
NVFP4 pruned build); about 28 GB wired on a Mac with Apple
silicon (macOS 26, Metal) or about 21 GB on an NVIDIA DGX Spark (CUDA 13). The full Boo stack needs about 79 GB on a Mac
(96 GB minimum, 128 GB or more recommended) or about 61 GB on a DGX Spark; see the GitHub README. Software: TensorFold
1.1.5 (the prebuilt builds in the GitHub release add Bolt and Cosmos3-Edge on Metal, Cosmos3-Edge on CUDA, and Drama 3
engines that start speaking sooner: on the Mac the keys 1.1.4-keys.1 engine with a voice-prefix cache that keeps every
voice, on the DGX Spark plotarmordev's NVFP4 fork with the same cache and the silence guard flags, because stock 1.1.5
refuses the pruned checkpoint on CUDA).
Full or Lite. Boo's optional lite voice, Kokoro-82M (hexgrad, Apache-2.0), replaces Drama 3; everything else is the same.
| DGX Spark, Full | DGX Spark, Lite | Mac, Full | Mac, Lite | |
|---|---|---|---|---|
| Platform | DGX Spark (GB10, 128 GB), Linux aarch64, CUDA 13 | same | Apple silicon, macOS 26+ | same |
| Minimum memory | 64 GB free | 48 GB free | 96 GB unified | 64 GB unified (tight) |
| Recommended memory | a whole DGX Spark | a whole DGX Spark, ≈ 40 GB left for other work | 128 GB or more | 96 GB or more |
| Free disk | ≈ 50 GB (≈ 57 GB while converting Cosmos) | ≈ 36 GB (≈ 45 GB while converting) | ≈ 50 GB (≈ 57 GB while converting) | ≈ 31 GB (≈ 40 GB while converting) |
| Measured in use, whole stack | ≈ 101 GB | ≈ 62 GB | ≈ 64 GB | ≈ 55 GB |
| Voice engine alone | Drama 3 ≈ 42 GB | Kokoro ≈ 3.5 GB | Drama 3 ≈ 10 GB | Kokoro ≈ 0.7 GB |
Lite mainly saves memory on the DGX Spark (about 39 GB); on a Mac only about 9 GB, because Bolt (≈ 45 GB), Cosmos3-Edge
and Parakeet set the floor. A 64 GB Mac with Lite is tight and untested: raise the GPU memory limit
(sudo sysctl iogpu.wired_limit_mb=57344, leaving macOS about 8 GB) and close other apps. Lite gives up these 14
voices, voice cloning, Drama 3's expressiveness and languages: it speaks English only, so no German and none of the
other European languages Drama 3 covers.
Install
The whole Boo stack, these voices included, installs with one command on a DGX Spark or an Apple silicon Mac:
curl -fsSL https://raw.githubusercontent.com/drowzeys/keys-ghost-machine-audio-visual-agent/main/install.sh | bash
It checks memory and disk, fetches the TensorFold 1.1.5 builds, the models and these voices, and starts Boo; kgm start,
kgm stop and kgm status run it after that.
Lite, for memory-constrained hardware (Kokoro-82M instead of Drama 3, no Drama 3 download):
# DGX Spark (Linux, GB10, CUDA 13), about 48 GB free memory
curl -fsSL https://raw.githubusercontent.com/drowzeys/keys-ghost-machine-audio-visual-agent/main/install.sh | bash -s -- --voice lite
# Mac (Apple silicon, macOS 26+), 96 GB comfortable, 64 GB tight
curl -fsSL https://raw.githubusercontent.com/drowzeys/keys-ghost-machine-audio-visual-agent/main/install.sh | bash -s -- --voice lite
Docker on a DGX Spark: -e VOICE_ENGINE=lite and no Drama 3 mount, with
ghcr.io/drowzeys/keys-ghost-machine-audio-visual-agent:0.3.0 (or both for the Full/Lite switch). --voice both installs both with a Full/Lite switch in the page; kgm voice full|lite|both switches later. Settings, the Docker alternative and uninstalling are in the
One-shot install section of the
GitHub README.
See it work
Real recordings of the page, hands-free. The voices are synthetic Drama 3 voices and the desk is a computer-made scene; no person appears. On TensorFold 1.1.5 (v0.3.0), voice-to-voice is 0.45–0.71 s on a Mac (M5 Ultra; 0.53–0.57 s with the lite voice) and 0.47–0.55 s on a DGX Spark, with Drama 3's first audio in Vesper at 60–80 ms on the Mac (warm, as in a conversation) and 121 ms on the DGX Spark.
Screenshots
| Phone (camera on) | Desktop |
|---|---|
 |
  |
A task handed to Hermes: the card shows only the tool's name while it works, then the answer.
The voices
| Voice | Description |
|---|---|
| vesper | British female; low, smooth and confident (default) |
| elara | British female; low, sleek and quick |
| lyra | British female; calm, smooth and assured |
| imogen | British female; warm, poised and articulate |
| seren | British female; light, elegant and lively |
| aria | British female; crisp, bright and polished |
| oliver | British male; warm and measured |
| skye | American female; low, smooth and steady |
| atlas | Deep, steady male narrator; unhurried storytelling |
| milo | Calm, low male voice; relaxed and reassuring |
| june | Warm female voice; natural and conversational |
| nova | Bright, energetic young female voice |
| sam | Friendly mid-range voice; clear, upbeat assistant |
| wren | Soft, gentle female voice; quiet and close |
previews/NAME.wav is each voice saying "Hi, I am NAME. Tell me what you would like to make today."
Lite voice (Kokoro-82M)
Boo's optional lite voice runs Kokoro-82M behind the same API as Drama 3 (MLX on the Mac, PyTorch CUDA on the DGX Spark). Measured on the live stacks:
| DGX Spark, lite | DGX Spark, full | Mac, lite | Mac, full | |
|---|---|---|---|---|
| Voice-to-voice, median | 0.33–0.40 s | 0.55–0.77 s | 0.49–0.64 s | 0.47–0.60 s |
| Voice engine memory in use | ≈ 3.5 GB | ≈ 42 GB | ≈ 0.7 GB | ≈ 10 GB |
| Whole stack in use | ≈ 62 GB | ≈ 101 GB | ≈ 55 GB | ≈ 64 GB |
Choose the full voice (Drama 3, these 14 voices) when memory allows; choose lite when memory or disk is short or for
the fastest replies on a DGX Spark. Lite speaks English only in Boo (American and British voices; Kokoro has no
German), with Kokoro's stock voices: bf_emma (British female, default), bf_isabella, bm_george, af_heart,
am_michael and more.
previews/kokoro/NAME.wav is a Kokoro voice saying "Hello, I am Boo, in the lite voice. I have queued three takes of
the fox shot, and the first one should be ready in about a minute." (synthetic, from Kokoro's stock voices): bf_emma,
bf_isabella, bm_george, af_heart, am_michael.
How they were made
Every voice is a take of Drama 3's own built-in voice, generated by the model; no recorded person is in any of
them. Drama 3 picks a different speaker for each random seed, so many takes of one line were generated and scored on
pitch, word error (Parakeet v3) and accent (SpeechBrain CommonAccent), and the chosen takes were encoded with Drama 3's
codec into reference codes. Each file says so in its source field.
Files
voices/mac/NAME.json:drama3-http-voice/1(frame-major codes, with a description), for the TensorFold Metal build with named voices.voices/spark/NAME.json:drama3-voice/1([10, frames]codes withcodes_sha256), for plotarmordev's TensorFold CUDA build.previews/NAME.wav: a short sample of each.previews/kokoro/NAME.wav: samples of the lite voice (Kokoro-82M stock voices; Apache-2.0).
Use
tensorfold serve DRAMA3_DIR --voices voices/mac --default-voice vesper # voices/spark on the CUDA build
curl -s http://127.0.0.1:8098/v1/audio/speech -H 'Content-Type: application/json' \
-d '{"model": "drama-3", "voice": "vesper", "input": "Hello there.", "response_format": "wav"}' -o hello.wav
Drama 3 sometimes goes quiet after the first sentence of a long line (about 1 in 5 paragraphs, the same input every
time; TensorFold#1275). On TensorFold 1.1.4 or later (and the DGX
Spark build since v0.2.0) add its silence guard to serve: --silence-floor-dbfs=-30 --silence-seconds=4 --silence-mid-s=2 --silence-mid-min-voiced-s=1 --silence-redraws=3 --silence-redraw-budget-s=90. Whole replies are redrawn; a stream ends at the silence instead, so
a streaming client should ask again for the unspoken text, as Boo does.
Licence
The voice codes are derived from Drama 3's output, so they fall under the Fish Audio Community License. Hosted services that generate speech for third parties need a separate licence from Fish Audio. The Kokoro previews are Kokoro-82M output (Apache-2.0).
Questions: GitHub Issues.
Credits
| Component | What it does in Boo | Author / org | Licence | Link |
|---|---|---|---|---|
| Nemotron 3.5 Bolt | The front brain: answers most turns, with mood tags for the face | NVIDIA | NVIDIA Open Model License | Hugging Face |
| Cosmos3-Edge | Camera questions and memory captions (its reasoner tower) | NVIDIA | OpenMDW 1.1 | Hugging Face |
| Parakeet TDT 0.6B v3 | Speech to text | NVIDIA | CC-BY-4.0 | Hugging Face |
| parakeet-mlx | Runs Parakeet on the Mac | senstella | Apache-2.0 | GitHub |
| NeMo | Runs Parakeet on the DGX Spark | NVIDIA | Apache-2.0 | GitHub |
| Drama 3 | The voice (speech synthesis); the 14 named voices are takes of its built-in voice | Fish Audio | Fish Audio Community License | Hugging Face |
| drama-3-NVFP4-Pruned, NVFP4 CUDA fork and named voices | The 4-bit pruned Drama 3 checkpoint, the DGX Spark's Drama engine and --voices |
plotarmordev | Fish Audio Community License (weights) · Apache-2.0 (engine) | checkpoint · fork |
| TensorFold | The engine for Bolt, Drama 3 and Cosmos3-Edge, on Metal and CUDA | Ash Hart | Apache-2.0 | GitHub |
| Kokoro-82M | The lite voice (optional, instead of Drama 3) | hexgrad | Apache-2.0 | Hugging Face |
| kokoro | Runs Kokoro-82M on PyTorch (DGX Spark) | hexgrad | Apache-2.0 | GitHub |
| mlx-audio | Runs Kokoro-82M on MLX (Mac); weights from mlx-community/Kokoro-82M-bf16 |
Prince Canuma (Blaizzy) | MIT | GitHub |
| misaki | Kokoro's text-to-phoneme step (G2P) | hexgrad | Apache-2.0 | GitHub |
| spaCy and en_core_web_sm | English tokenising and tagging inside misaki | Explosion | MIT | GitHub |
| espeak-ng (through phonemizer-fork and espeakng-loader) | misaki's fallback pronunciation for words its dictionary lacks | espeak-ng contributors | GPL-3.0 | GitHub |
| StyleTTS 2 | The architecture Kokoro-82M is built on (with an iSTFTNet decoder) | Yinghao Aaron Li et al. | MIT | GitHub |
| Eyes of Kai | The continuity log: timestamped captions so Boo can answer about the past | tonyd2wild | MIT | GitHub |
| Hermes Agent | The back agent with tools and skills, over ACP | Nous Research | see repository | GitHub |
| qwen-audio-agent | Design inspiration: the frontstage/backstage split (a fast voice in front, a tool-using agent behind that reports back); no code from it is used | Qwen Audio team | Apache-2.0 | GitHub |
| RED-SNOW 5.3 Flash | The back agent's model in the reference setup (a GLM-5.3-Flash build); any OpenAI-compatible model works | RED-SNOW | see model card | — |
| Silero VAD v5 | Hears when someone is speaking (silero_vad.onnx ships in the GitHub repo) |
Silero | MIT | GitHub |
| MediaPipe Face Detector and Hand Landmarker | Gaze tracking, exact finger counts and which hand, in the page | Apache-2.0 | GitHub | |
| Three.js | Boo, the 3D ghost | three.js authors (mrdoob) | MIT | GitHub |
| ddgs | Web search for quick live facts | deedy5 | MIT | GitHub |
| wttr.in | Weather for quick live facts (data from WorldWeatherOnline) | Igor Chubin | Apache-2.0 | GitHub |
| NVIDIA PyTorch container | The Docker image's base (nvcr.io/nvidia/pytorch:26.07-py3) |
NVIDIA | NVIDIA Deep Learning Container License | NGC |
| Qwen3.5-0.8B (earlier versions) | The first memory captioner | Qwen team, Alibaba Cloud | Apache-2.0 | Hugging Face |
| AEON-7 Nemotron 3 Nano Omni build (earlier versions) | The first camera model, from NVIDIA's Nemotron 3 Nano Omni | AEON-7 | NVIDIA Open Model License | Hugging Face |
Full details: CREDITS.md
Model tree for drowzeys/keys-ghost-machine-audio-visual-agent
Base model
fishaudio/drama-3