| --- |
| license: other |
| license_name: fish-audio-community-license |
| license_link: https://huggingface.co/fishaudio/drama-3 |
| base_model: fishaudio/drama-3 |
| pipeline_tag: text-to-speech |
| language: |
| - en |
| tags: |
| - drama-3 |
| - tts |
| - voices |
| - tensorfold |
| - fish-audio |
| - kokoro |
| --- |
| |
| # Keys Ghost Machine — audio/visual agent, powered by [TensorFold](https://github.com/ashhart/TensorFold) |
|
|
| <video controls autoplay muted loop playsinline width="320" poster="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-45s-poster.jpg" src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-45s.mp4"></video> |
|
|
| The voices (Drama 3), the promo and screenshots for Boo, and previews of its optional lite voice (Kokoro-82M). |
|
|
| Fourteen named voices for [Fish Audio Drama 3](https://huggingface.co/fishaudio/drama-3), made for **Boo**, the |
| audio/visual agent in [drowzeys/keys-ghost-machine-audio-visual-agent](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent). |
| Vesper is Boo's default. |
|
|
| ## What's new in v0.3.0 |
|
|
| **Boo moves to TensorFold 1.1.5** (the voices here are unchanged). Bolt on a DGX Spark runs on the stock 1.1.5 release; |
| Bolt and Cosmos3-Edge on the Mac, and Cosmos3-Edge on CUDA, run on the keys 1.1.5 builds. To update Boo, run the install |
| command again, then `kgm restart`; in Docker, pull image `0.3.0`. |
|
|
| - **Same answers, same speed:** on an M5 Ultra Bolt's greedy replies and Cosmos3-Edge's camera answers are identical to |
| the 1.1.4 builds (7 of 7 each) at the same speed (Bolt first token 77–81 ms, Cosmos3-Edge 78 ms). |
| - **Drama 3 on the Mac stays on the keys 1.1.4-keys.1 engine** (shipped beside 1.1.5 in the Mac archive). Stock 1.1.5 |
| reads the NVFP4 pruned checkpoint on Metal and reuses a voice's reference prefill, but its first audio in a named voice |
| was 107–343 ms against a steady 67–79 ms for the keys engine, so these voices keep speaking from the faster one. The |
| DGX Spark's Drama 3 is unchanged too, and the v0.2.2 fixes (no keep-warm line on the Spark, cuBLASLt 13.1 in Docker) |
| stay. |
|
|
| ### v0.2.2 |
|
|
| **Boo's Full voice (Drama 3) on a DGX Spark answers about twice as fast: 0.47–0.55 s voice to voice instead of |
| 1.0–1.1 s** (the voices here are unchanged). To update Boo, run the install command again, then `kgm restart`; in |
| Docker, pull image `0.2.2`. |
|
|
| - **No Drama 3 keep-warm line on the DGX Spark:** its CUDA Drama 3 runs one synthesis at a time, so the throwaway |
| "Mm." Boo spoke while you talked made the reply wait. The Spark profile now turns it off (Bolt stays warm). |
| - **Docker:** the TensorFold engines get the cuBLASLt 13.1 they are qualified with, so Drama 3's fast one-row kernel is |
| on again. |
|
|
| ### v0.2.1 |
|
|
| A fix release for v0.2.0 (superseded by v0.2.2). |
|
|
| - **Fresh DGX Spark installs: the page could not connect** after Parakeet warmed up. NeMo's CUDA-graph decoder wrote a |
| NUL into Python's shared `b" "` object (an empty NVRTC log read into `b" " * 1`), which broke uvicorn's status lines |
| and websockets' parser and left damaged `.pyc` files. Boo now gives NVRTC its own buffer (the Docker image was not |
| affected). |
| - **"(Boo says)" at the start of a reply:** after a switch of voice engine (Full/Lite) or voice, the sample line "Hi, |
| this is how I sound now." went into the conversation as "(Boo says) Hi, this is how I sound now.", and Bolt could |
| copy the label, or the whole line, into its next replies. The sample line is no longer kept in the conversation, |
| Boo's own short lines are kept without a label, and a reply that still starts with a stage direction such as |
| "(Boo says)" loses it before it is spoken or stored. |
| - **Docs:** the Mac's Drama 3 first audio is ~60 ms on TensorFold 1.1.4 with the engine warm, as in a conversation |
| (it said 122 ms, the figure after the engine has sat idle). |
|
|
| ### New in v0.2.0: Kokoro Lite, a second voice for smaller machines |
|
|
| [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) |
| (hexgrad, Apache-2.0) can now speak for Boo instead of Drama 3, or beside it with a switch in the page. It is for |
| memory-constrained hardware: a DGX Spark that shares its memory with other work, or a Mac at the low end of the |
| [requirements](#requirements). Everything else (Bolt, Cosmos3-Edge, Parakeet, Eyes of Kai, Hermes) stays the same. |
|
|
| - **Install Lite, DGX Spark** (about 48 GB free memory): |
| `curl -fsSL https://raw.githubusercontent.com/drowzeys/keys-ghost-machine-audio-visual-agent/main/install.sh | bash -s -- --voice lite` |
| - **Install Lite, Mac** (Apple silicon, macOS 26+; 96 GB comfortable, 64 GB tight): the same command, |
| `curl -fsSL https://raw.githubusercontent.com/drowzeys/keys-ghost-machine-audio-visual-agent/main/install.sh | bash -s -- --voice lite` |
| - **Docker, Lite** (DGX Spark, no Drama 3 mount needed): |
| ```sh |
| docker run --gpus all --rm -p 8095:8095 -e VOICE_ENGINE=lite -e KAI=1 \ |
| -v ~/keys-ghost-machine/models/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB:/models/bolt \ |
| -v ~/keys-ghost-machine/models/cosmos3-tf:/models/cosmos3 -v ~/models/hf-cache:/models/hf-cache \ |
| ghcr.io/drowzeys/keys-ghost-machine-audio-visual-agent:0.3.0 |
| ``` |
| - **Both, with a Full/Lite toggle:** install with `--voice both` (or `-e VOICE_ENGINE=both` in Docker, with the |
| Drama 3 mount) and the page shows a **Full / Lite** switch per conversation; the voice list follows the engine. |
| On an existing install, `kgm voice full|lite|both` switches. Pick a Kokoro voice in the page's Voice list |
| (`bf_emma` by default, `af_heart` is Kokoro's best rated) or set `KOKORO_VOICE=af_heart` in Docker. |
| - **Trade-off:** on a DGX Spark Lite answers in about 0.33–0.40 s voice to voice (Full 0.55–0.77 s) and uses about |
| 40 GB less memory. On a Mac it is about as fast as Full and saves about 9 GB. Lite speaks English only in Boo and |
| has no voice cloning, emotion control or the 14 Drama 3 voices. |
|
|
| **Kokoro Lite vs Drama 3 (4-bit)** |
|
|
| | | Kokoro Lite | Drama 3 4-bit (NVFP4) | |
| |---|---|---| |
| | Model | 82M parameters, ~330 MB | Billions of parameters, 4-bit NVFP4 | |
| | Voice-engine memory: DGX Spark | ~3.5 GB | ~42 GB | |
| | Voice-engine memory: Mac | ~0.8 GB | ~10 GB | |
| | Voice starts: Spark | 33–40 ms | 112–121 ms | |
| | Voice starts: Mac | 51–54 ms | ~60 ms | |
| | Voice to voice: Spark | 0.33–0.40 s | 0.55–0.77 s | |
| | Voice to voice: Mac | 0.49–0.64 s | 0.43–0.61 s | |
| | Naturalness (UTMOS) | 4.35 (bf_emma) – 4.52 (af_heart) | 4.53 (Vesper) | |
| | Voices | 54 built in | 14 custom, incl. Vesper | |
| | Voice cloning | No | Yes | |
| | Emotion/style control | No | Yes | |
| | Long-line stalls | None | Occasional; fixed by the silence guard + resume in v0.2.0 | |
| | Streaming | Whole sentence (<0.13 s) | Streams within a sentence | |
| | Languages | EN, ES, FR, IT, PT, HI, JA, ZH | Wider, incl. German | |
| | Licence | Apache-2.0 | Fish Audio Community License (hosted commercial use needs a Fish licence) | |
|
|
| Spark: Lite is twice as fast and frees ~40 GB. Mac: same speed, so run `both` and keep Vesper with a Lite toggle. |
| Choose Drama for a signature voice, cloning, emotion or German; choose Kokoro for small or shared hardware, an Apache |
| licence or the lowest delay. |
|
|
| **Fixes in v0.2.0** |
|
|
| - **Drama 3 long-line stalls:** a long reply that went quiet after its first sentence (on about 1 in 5 paragraph-length |
| lines) is now caught. The DGX Spark's Drama 3 build gains TensorFold 1.1.4's silence guard (new in the v0.2.0 |
| Linux archive and the Docker image), the Mac build already had it, and Boo itself drops a stalled take's quiet tail |
| and asks for the unspoken rest. |
| - **Never deaf after a fast short reply:** a quick reply such as "I'm on it." could leave Boo marked as still |
| speaking, and the session stopped listening. Fixed (64 of 64 turns heard in testing). |
| - **Drama 3 "speech output capacity is full" (503):** the keep-warm loop no longer outlives its page or a stopped |
| microphone (it stops on disconnect, after 2 s without audio and after 60 s), warms with one "Mm." at a time and never |
| while anything is being said; a refused line is asked again for up to `retry_s` (6 s) instead of being lost; a |
| stopped turn frees its speech slot at once. |
| - **Bolt repetition loops:** a reply that starts repeating a sentence it already said ends there, unsaid. |
| - **Interrupted replies** stopped before their first word no longer leave "(interrupted)" in the history for the |
| brain to copy. |
| - The reference Mac now runs `voice_engine: both`, with the Full/Lite toggle tested live. |
|
|
| Builds: [v0.3.0 release](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/releases/tag/v0.3.0) · |
| `docker pull ghcr.io/drowzeys/keys-ghost-machine-audio-visual-agent:0.3.0` |
|
|
| ## Requirements |
|
|
| Drama 3 served by TensorFold with named voices (`--voices`): about 21 GB of weights (the |
| [NVFP4 pruned build](https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned)); about 28 GB wired on a Mac with Apple |
| silicon (macOS 26, Metal) or about 21 GB on an NVIDIA DGX Spark (CUDA 13). The full Boo stack needs about 79 GB on a Mac |
| (96 GB minimum, 128 GB or more recommended) or about 61 GB on a DGX Spark; see the GitHub README. Software: TensorFold |
| 1.1.5 (the prebuilt builds in the GitHub release add Bolt and Cosmos3-Edge on Metal, Cosmos3-Edge on CUDA, and Drama 3 |
| engines that start speaking sooner: on the Mac the keys 1.1.4-keys.1 engine with a voice-prefix cache that keeps every |
| voice, on the DGX Spark plotarmordev's NVFP4 fork with the same cache and the silence guard flags, because stock 1.1.5 |
| refuses the pruned checkpoint on CUDA). |
|
|
| **Full or Lite.** Boo's optional lite voice, [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) (hexgrad, |
| Apache-2.0), replaces Drama 3; everything else is the same. |
|
|
| | | DGX Spark, Full | DGX Spark, Lite | Mac, Full | Mac, Lite | |
| |---|---|---|---|---| |
| | Platform | DGX Spark (GB10, 128 GB), Linux aarch64, CUDA 13 | same | Apple silicon, macOS 26+ | same | |
| | Minimum memory | 64 GB free | **48 GB free** | 96 GB unified | **64 GB unified** (tight) | |
| | Recommended memory | a whole DGX Spark | a whole DGX Spark, ≈ 40 GB left for other work | 128 GB or more | 96 GB or more | |
| | Free disk | ≈ 50 GB (≈ 57 GB while converting Cosmos) | ≈ 36 GB (≈ 45 GB while converting) | ≈ 50 GB (≈ 57 GB while converting) | ≈ 31 GB (≈ 40 GB while converting) | |
| | Measured in use, whole stack | ≈ 101 GB | ≈ 62 GB | ≈ 64 GB | ≈ 55 GB | |
| | Voice engine alone | Drama 3 ≈ 42 GB | Kokoro ≈ 3.5 GB | Drama 3 ≈ 10 GB | Kokoro ≈ 0.7 GB | |
|
|
| Lite mainly saves memory on the DGX Spark (about 39 GB); on a Mac only about 9 GB, because Bolt (≈ 45 GB), Cosmos3-Edge |
| and Parakeet set the floor. A 64 GB Mac with Lite is tight and untested: raise the GPU memory limit |
| (`sudo sysctl iogpu.wired_limit_mb=57344`, leaving macOS about 8 GB) and close other apps. Lite gives up these 14 |
| voices, voice cloning, Drama 3's expressiveness and languages: it speaks English only, so no German and none of the |
| other European languages Drama 3 covers. |
|
|
| ## Install |
|
|
| The whole Boo stack, these voices included, installs with one command on a DGX Spark or an Apple silicon Mac: |
|
|
| ```sh |
| curl -fsSL https://raw.githubusercontent.com/drowzeys/keys-ghost-machine-audio-visual-agent/main/install.sh | bash |
| ``` |
|
|
| It checks memory and disk, fetches the TensorFold 1.1.5 builds, the models and these voices, and starts Boo; `kgm start`, |
| `kgm stop` and `kgm status` run it after that. |
|
|
| **Lite, for memory-constrained hardware** (Kokoro-82M instead of Drama 3, no Drama 3 download): |
|
|
| ```sh |
| # DGX Spark (Linux, GB10, CUDA 13), about 48 GB free memory |
| curl -fsSL https://raw.githubusercontent.com/drowzeys/keys-ghost-machine-audio-visual-agent/main/install.sh | bash -s -- --voice lite |
| # Mac (Apple silicon, macOS 26+), 96 GB comfortable, 64 GB tight |
| curl -fsSL https://raw.githubusercontent.com/drowzeys/keys-ghost-machine-audio-visual-agent/main/install.sh | bash -s -- --voice lite |
| ``` |
|
|
| Docker on a DGX Spark: `-e VOICE_ENGINE=lite` and no Drama 3 mount, with |
| `ghcr.io/drowzeys/keys-ghost-machine-audio-visual-agent:0.3.0` (or `both` for the Full/Lite switch). `--voice both` installs both with a Full/Lite switch in the page; `kgm |
| voice full|lite|both` switches later. Settings, the Docker alternative and uninstalling are in the |
| [One-shot install](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent#one-shot-install) section of the |
| GitHub README. |
|
|
| ## See it work |
|
|
| <video controls width="100%" poster="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-poster.jpg" src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-landscape.mp4"></video> |
|
|
| Real recordings of the page, hands-free. The voices are synthetic Drama 3 voices and the desk is a computer-made scene; |
| no person appears. On TensorFold 1.1.5 (v0.3.0), voice-to-voice is 0.45–0.71 s on a Mac (M5 Ultra; 0.53–0.57 s with |
| the lite voice) and 0.47–0.55 s on a DGX Spark, with Drama 3's first audio in Vesper at 60–80 ms on the Mac (warm, as |
| in a conversation) and 121 ms on the DGX Spark. |
|
|
| ## Screenshots |
|
|
| | Phone (camera on) | Desktop | |
| |---|---| |
| | <img src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/ui-phone-camera.png" width="300" alt="Boo on a phone with the camera on, describing a desk with a WOW mug"> | <img src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/ui-desktop.png" width="560" alt="Boo on a desktop with its settings and a live-fact answer"><br><br><img src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/ui-desktop-light.png" width="560" alt="The same page in the light theme"> | |
|
|
| A task handed to Hermes: the card shows only the tool's name while it works, then the answer. |
|
|
| <img src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/ui-hermes-card.png" width="640" alt="A Hermes task card showing terminal while working, then the answer"> |
|
|
| ## The voices |
|
|
| | Voice | Description | |
| |---|---| |
| | vesper | British female; low, smooth and confident (default) | |
| | elara | British female; low, sleek and quick | |
| | lyra | British female; calm, smooth and assured | |
| | imogen | British female; warm, poised and articulate | |
| | seren | British female; light, elegant and lively | |
| | aria | British female; crisp, bright and polished | |
| | oliver | British male; warm and measured | |
| | skye | American female; low, smooth and steady | |
| | atlas | Deep, steady male narrator; unhurried storytelling | |
| | milo | Calm, low male voice; relaxed and reassuring | |
| | june | Warm female voice; natural and conversational | |
| | nova | Bright, energetic young female voice | |
| | sam | Friendly mid-range voice; clear, upbeat assistant | |
| | wren | Soft, gentle female voice; quiet and close | |
|
|
| `previews/NAME.wav` is each voice saying "Hi, I am NAME. Tell me what you would like to make today." |
|
|
| ## Lite voice (Kokoro-82M) |
|
|
| Boo's optional lite voice runs [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) behind the same API as Drama 3 |
| (MLX on the Mac, PyTorch CUDA on the DGX Spark). Measured on the live stacks: |
|
|
| | | DGX Spark, lite | DGX Spark, full | Mac, lite | Mac, full | |
| |---|---|---|---|---| |
| | Voice-to-voice, median | **0.33–0.40 s** | 0.55–0.77 s | 0.49–0.64 s | 0.47–0.60 s | |
| | Voice engine memory in use | **≈ 3.5 GB** | ≈ 42 GB | **≈ 0.7 GB** | ≈ 10 GB | |
| | Whole stack in use | **≈ 62 GB** | ≈ 101 GB | **≈ 55 GB** | ≈ 64 GB | |
|
|
| Choose the full voice (Drama 3, these 14 voices) when memory allows; choose lite when memory or disk is short or for |
| the fastest replies on a DGX Spark. Lite speaks English only in Boo (American and British voices; Kokoro has no |
| German), with Kokoro's stock voices: `bf_emma` (British female, default), `bf_isabella`, `bm_george`, `af_heart`, |
| `am_michael` and more. |
|
|
| `previews/kokoro/NAME.wav` is a Kokoro voice saying "Hello, I am Boo, in the lite voice. I have queued three takes of |
| the fox shot, and the first one should be ready in about a minute." (synthetic, from Kokoro's stock voices): `bf_emma`, |
| `bf_isabella`, `bm_george`, `af_heart`, `am_michael`. |
|
|
| ## How they were made |
|
|
| Every voice is a take of Drama 3's own built-in voice, generated by the model; **no recorded person is in any of |
| them**. Drama 3 picks a different speaker for each random seed, so many takes of one line were generated and scored on |
| pitch, word error (Parakeet v3) and accent (SpeechBrain CommonAccent), and the chosen takes were encoded with Drama 3's |
| codec into reference codes. Each file says so in its `source` field. |
|
|
| ## Files |
|
|
| - `voices/mac/NAME.json`: `drama3-http-voice/1` (frame-major codes, with a description), for the TensorFold Metal build |
| with named voices. |
| - `voices/spark/NAME.json`: `drama3-voice/1` (`[10, frames]` codes with `codes_sha256`), for |
| [plotarmordev's TensorFold](https://github.com/plotarmordev/TensorFold) CUDA build. |
| - `previews/NAME.wav`: a short sample of each. |
| - `previews/kokoro/NAME.wav`: samples of the lite voice (Kokoro-82M stock voices; Apache-2.0). |
|
|
| ## Use |
|
|
| ```sh |
| tensorfold serve DRAMA3_DIR --voices voices/mac --default-voice vesper # voices/spark on the CUDA build |
| curl -s http://127.0.0.1:8098/v1/audio/speech -H 'Content-Type: application/json' \ |
| -d '{"model": "drama-3", "voice": "vesper", "input": "Hello there.", "response_format": "wav"}' -o hello.wav |
| ``` |
|
|
| Drama 3 sometimes goes quiet after the first sentence of a long line (about 1 in 5 paragraphs, the same input every |
| time; [TensorFold#1275](https://github.com/ashhart/TensorFold/issues/1275)). On TensorFold 1.1.4 or later (and the DGX |
| Spark build since v0.2.0) add its silence guard to `serve`: `--silence-floor-dbfs=-30 --silence-seconds=4 --silence-mid-s=2 --silence-mid-min-voiced-s=1 |
| --silence-redraws=3 --silence-redraw-budget-s=90`. Whole replies are redrawn; a stream ends at the silence instead, so |
| a streaming client should ask again for the unspoken text, as Boo does. |
|
|
| ## Licence |
|
|
| The voice codes are derived from Drama 3's output, so they fall under the |
| [Fish Audio Community License](https://huggingface.co/fishaudio/drama-3). Hosted services that generate speech for third |
| parties need a separate licence from Fish Audio. The Kokoro previews are Kokoro-82M output (Apache-2.0). |
|
|
| Questions: [GitHub Issues](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/issues). |
|
|
| ## Credits |
|
|
| | Component | What it does in Boo | Author / org | Licence | Link | |
| |---|---|---|---|---| |
| | Nemotron 3.5 Bolt | The front brain: answers most turns, with mood tags for the face | NVIDIA | NVIDIA Open Model License | [Hugging Face](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB) | |
| | Cosmos3-Edge | Camera questions and memory captions (its reasoner tower) | NVIDIA | OpenMDW 1.1 | [Hugging Face](https://huggingface.co/nvidia/Cosmos3-Edge) | |
| | Parakeet TDT 0.6B v3 | Speech to text | NVIDIA | CC-BY-4.0 | [Hugging Face](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) | |
| | parakeet-mlx | Runs Parakeet on the Mac | senstella | Apache-2.0 | [GitHub](https://github.com/senstella/parakeet-mlx) | |
| | NeMo | Runs Parakeet on the DGX Spark | NVIDIA | Apache-2.0 | [GitHub](https://github.com/NVIDIA/NeMo) | |
| | Drama 3 | The voice (speech synthesis); the 14 named voices are takes of its built-in voice | Fish Audio | Fish Audio Community License | [Hugging Face](https://huggingface.co/fishaudio/drama-3) | |
| | drama-3-NVFP4-Pruned, NVFP4 CUDA fork and named voices | The 4-bit pruned Drama 3 checkpoint, the DGX Spark's Drama engine and `--voices` | plotarmordev | Fish Audio Community License (weights) · Apache-2.0 (engine) | [checkpoint](https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned) · [fork](https://github.com/plotarmordev/TensorFold) | |
| | TensorFold | The engine for Bolt, Drama 3 and Cosmos3-Edge, on Metal and CUDA | Ash Hart | Apache-2.0 | [GitHub](https://github.com/ashhart/TensorFold) | |
| | Kokoro-82M | The lite voice (optional, instead of Drama 3) | hexgrad | Apache-2.0 | [Hugging Face](https://huggingface.co/hexgrad/Kokoro-82M) | |
| | kokoro | Runs Kokoro-82M on PyTorch (DGX Spark) | hexgrad | Apache-2.0 | [GitHub](https://github.com/hexgrad/kokoro) | |
| | mlx-audio | Runs Kokoro-82M on MLX (Mac); weights from `mlx-community/Kokoro-82M-bf16` | Prince Canuma (Blaizzy) | MIT | [GitHub](https://github.com/Blaizzy/mlx-audio) | |
| | misaki | Kokoro's text-to-phoneme step (G2P) | hexgrad | Apache-2.0 | [GitHub](https://github.com/hexgrad/misaki) | |
| | spaCy and en_core_web_sm | English tokenising and tagging inside misaki | Explosion | MIT | [GitHub](https://github.com/explosion/spaCy) | |
| | espeak-ng (through phonemizer-fork and espeakng-loader) | misaki's fallback pronunciation for words its dictionary lacks | espeak-ng contributors | GPL-3.0 | [GitHub](https://github.com/espeak-ng/espeak-ng) | |
| | StyleTTS 2 | The architecture Kokoro-82M is built on (with an iSTFTNet decoder) | Yinghao Aaron Li et al. | MIT | [GitHub](https://github.com/yl4579/StyleTTS2) | |
| | Eyes of Kai | The continuity log: timestamped captions so Boo can answer about the past | tonyd2wild | MIT | [GitHub](https://github.com/tonyd2wild/eyes-of-kai) | |
| | Hermes Agent | The back agent with tools and skills, over ACP | Nous Research | see repository | [GitHub](https://github.com/NousResearch/hermes-agent) | |
| | qwen-audio-agent | Design inspiration: the frontstage/backstage split (a fast voice in front, a tool-using agent behind that reports back); no code from it is used | Qwen Audio team | Apache-2.0 | [GitHub](https://github.com/QwenAudio/qwen-audio-agent) | |
| | RED-SNOW 5.3 Flash | The back agent's model in the reference setup (a GLM-5.3-Flash build); any OpenAI-compatible model works | RED-SNOW | see model card | — | |
| | Silero VAD v5 | Hears when someone is speaking (`silero_vad.onnx` ships in the GitHub repo) | Silero | MIT | [GitHub](https://github.com/snakers4/silero-vad) | |
| | MediaPipe Face Detector and Hand Landmarker | Gaze tracking, exact finger counts and which hand, in the page | Google | Apache-2.0 | [GitHub](https://github.com/google-ai-edge/mediapipe) | |
| | Three.js | Boo, the 3D ghost | three.js authors (mrdoob) | MIT | [GitHub](https://github.com/mrdoob/three.js) | |
| | ddgs | Web search for quick live facts | deedy5 | MIT | [GitHub](https://github.com/deedy5/ddgs) | |
| | wttr.in | Weather for quick live facts (data from WorldWeatherOnline) | Igor Chubin | Apache-2.0 | [GitHub](https://github.com/chubin/wttr.in) | |
| | NVIDIA PyTorch container | The Docker image's base (`nvcr.io/nvidia/pytorch:26.07-py3`) | NVIDIA | NVIDIA Deep Learning Container License | [NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch) | |
| | Qwen3.5-0.8B (earlier versions) | The first memory captioner | Qwen team, Alibaba Cloud | Apache-2.0 | [Hugging Face](https://huggingface.co/Qwen) | |
| | AEON-7 Nemotron 3 Nano Omni build (earlier versions) | The first camera model, from NVIDIA's Nemotron 3 Nano Omni | AEON-7 | NVIDIA Open Model License | [Hugging Face](https://huggingface.co/AEON-7) | |
|
|
| Full details: [CREDITS.md](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/blob/main/CREDITS.md) |
|
|
