drowzeys commited on
Commit
5e38e47
·
verified ·
1 Parent(s): 15489fe

Card: powered-by-TensorFold title, promo video, screenshots, TensorFold 1.1.4 notes, Credits table

Browse files
.gitattributes CHANGED
@@ -47,3 +47,9 @@ previews/seren.wav filter=lfs diff=lfs merge=lfs -text
47
  previews/skye.wav filter=lfs diff=lfs merge=lfs -text
48
  previews/vesper.wav filter=lfs diff=lfs merge=lfs -text
49
  previews/wren.wav filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
47
  previews/skye.wav filter=lfs diff=lfs merge=lfs -text
48
  previews/vesper.wav filter=lfs diff=lfs merge=lfs -text
49
  previews/wren.wav filter=lfs diff=lfs merge=lfs -text
50
+ media/promo-45s.mp4 filter=lfs diff=lfs merge=lfs -text
51
+ media/promo-landscape.mp4 filter=lfs diff=lfs merge=lfs -text
52
+ media/ui-desktop-light.png filter=lfs diff=lfs merge=lfs -text
53
+ media/ui-desktop.png filter=lfs diff=lfs merge=lfs -text
54
+ media/ui-hermes-card.png filter=lfs diff=lfs merge=lfs -text
55
+ media/ui-phone-camera.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -14,7 +14,9 @@ tags:
14
  - fish-audio
15
  ---
16
 
17
- # keys-ghost-machine voices (Drama 3)
 
 
18
 
19
  Fourteen named voices for [Fish Audio Drama 3](https://huggingface.co/fishaudio/drama-3), made for **Boo**, the
20
  audio/visual agent in [drowzeys/keys-ghost-machine-audio-visual-agent](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent).
@@ -25,7 +27,29 @@ Vesper is Boo's default.
25
  Drama 3 served by TensorFold with named voices (`--voices`): about 21 GB of weights (the
26
  [NVFP4 pruned build](https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned)); about 28 GB wired on a Mac with Apple
27
  silicon (macOS 26, Metal) or about 21 GB on an NVIDIA DGX Spark (CUDA 13). The full Boo stack needs about 79 GB on a Mac
28
- (96 GB minimum, 128 GB or more recommended) or about 61 GB on a DGX Spark; see the GitHub README.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
 
30
  ## The voices
31
 
@@ -75,7 +99,33 @@ curl -s http://127.0.0.1:8098/v1/audio/speech -H 'Content-Type: application/json
75
 
76
  The voice codes are derived from Drama 3's output, so they fall under the
77
  [Fish Audio Community License](https://huggingface.co/fishaudio/drama-3). Hosted services that generate speech for third
78
- parties need a separate licence from Fish Audio. Credits for everything Boo uses are in the GitHub repository's
79
- CREDITS.md.
80
 
81
  Questions: [GitHub Issues](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/issues).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  - fish-audio
15
  ---
16
 
17
+ # Keys Ghost Machine — audio/visual agent, powered by [TensorFold](https://github.com/ashhart/TensorFold)
18
+
19
+ The voices (Drama 3), the promo and screenshots for Boo.
20
 
21
  Fourteen named voices for [Fish Audio Drama 3](https://huggingface.co/fishaudio/drama-3), made for **Boo**, the
22
  audio/visual agent in [drowzeys/keys-ghost-machine-audio-visual-agent](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent).
 
27
  Drama 3 served by TensorFold with named voices (`--voices`): about 21 GB of weights (the
28
  [NVFP4 pruned build](https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned)); about 28 GB wired on a Mac with Apple
29
  silicon (macOS 26, Metal) or about 21 GB on an NVIDIA DGX Spark (CUDA 13). The full Boo stack needs about 79 GB on a Mac
30
+ (96 GB minimum, 128 GB or more recommended) or about 61 GB on a DGX Spark; see the GitHub README. Software: TensorFold
31
+ 1.1.4 (the prebuilt builds in the GitHub release add NVFP4 pruned Drama 3, the voice-prefix cache and Cosmos3-Edge; the
32
+ DGX Spark's Drama 3 runs on plotarmordev's NVFP4 fork, because stock 1.1.4 refuses the pruned checkpoint).
33
+
34
+ ## See it work
35
+
36
+ <video controls width="100%" poster="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-poster.jpg" src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-landscape.mp4"></video>
37
+
38
+ Phone cut (46 s): <video controls width="320" src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-45s.mp4"></video>
39
+
40
+ Real recordings of the page, hands-free. The voices are synthetic Drama 3 voices and the desk is a computer-made scene;
41
+ no person appears. On TensorFold 1.1.4, voice-to-voice is 0.47–0.60 s on a Mac (M5 Ultra) and 0.55–0.77 s on a DGX
42
+ Spark, with Drama 3's first audio in Vesper at 122 ms and 121 ms.
43
+
44
+ ## Screenshots
45
+
46
+ | Phone (camera on) | Desktop |
47
+ |---|---|
48
+ | <img src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/ui-phone-camera.png" width="300" alt="Boo on a phone with the camera on, describing a desk with a WOW mug"> | <img src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/ui-desktop.png" width="560" alt="Boo on a desktop with its settings and a live-fact answer"><br><br><img src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/ui-desktop-light.png" width="560" alt="The same page in the light theme"> |
49
+
50
+ A task handed to Hermes: the card shows only the tool's name while it works, then the answer.
51
+
52
+ <img src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/ui-hermes-card.png" width="640" alt="A Hermes task card showing terminal while working, then the answer">
53
 
54
  ## The voices
55
 
 
99
 
100
  The voice codes are derived from Drama 3's output, so they fall under the
101
  [Fish Audio Community License](https://huggingface.co/fishaudio/drama-3). Hosted services that generate speech for third
102
+ parties need a separate licence from Fish Audio.
 
103
 
104
  Questions: [GitHub Issues](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/issues).
105
+
106
+ ## Credits
107
+
108
+ | Component | What it does in Boo | Author / org | Licence | Link |
109
+ |---|---|---|---|---|
110
+ | Nemotron 3.5 Bolt | The front brain: answers most turns, with mood tags for the face | NVIDIA | NVIDIA Open Model License | [Hugging Face](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB) |
111
+ | Cosmos3-Edge | Camera questions and memory captions (its reasoner tower) | NVIDIA | OpenMDW 1.1 | [Hugging Face](https://huggingface.co/nvidia/Cosmos3-Edge) |
112
+ | Parakeet TDT 0.6B v3 | Speech to text | NVIDIA | CC-BY-4.0 | [Hugging Face](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) |
113
+ | parakeet-mlx | Runs Parakeet on the Mac | senstella | Apache-2.0 | [GitHub](https://github.com/senstella/parakeet-mlx) |
114
+ | NeMo | Runs Parakeet on the DGX Spark | NVIDIA | Apache-2.0 | [GitHub](https://github.com/NVIDIA/NeMo) |
115
+ | Drama 3 | The voice (speech synthesis); the 14 named voices are takes of its built-in voice | Fish Audio | Fish Audio Community License | [Hugging Face](https://huggingface.co/fishaudio/drama-3) |
116
+ | drama-3-NVFP4-Pruned, NVFP4 CUDA fork and named voices | The 4-bit pruned Drama 3 checkpoint, the DGX Spark's Drama engine and `--voices` | plotarmordev | Fish Audio Community License (weights) · Apache-2.0 (engine) | [checkpoint](https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned) · [fork](https://github.com/plotarmordev/TensorFold) |
117
+ | TensorFold | The engine for Bolt, Drama 3 and Cosmos3-Edge, on Metal and CUDA | Ash Hart | Apache-2.0 | [GitHub](https://github.com/ashhart/TensorFold) |
118
+ | Eyes of Kai | The continuity log: timestamped captions so Boo can answer about the past | tonyd2wild | MIT | [GitHub](https://github.com/tonyd2wild/eyes-of-kai) |
119
+ | Hermes Agent | The back agent with tools and skills, over ACP | Nous Research | see repository | [GitHub](https://github.com/NousResearch/hermes-agent) |
120
+ | qwen-audio-agent | Design inspiration: the frontstage/backstage split (a fast voice in front, a tool-using agent behind that reports back); no code from it is used | Qwen Audio team | Apache-2.0 | [GitHub](https://github.com/QwenAudio/qwen-audio-agent) |
121
+ | RED-SNOW 5.3 Flash | The back agent's model in the reference setup (a GLM-5.3-Flash build); any OpenAI-compatible model works | RED-SNOW | see model card | — |
122
+ | Silero VAD v5 | Hears when someone is speaking (`silero_vad.onnx` ships here) | Silero | MIT | [GitHub](https://github.com/snakers4/silero-vad) |
123
+ | MediaPipe Face Detector and Hand Landmarker | Gaze tracking, exact finger counts and which hand, in the page | Google | Apache-2.0 | [GitHub](https://github.com/google-ai-edge/mediapipe) |
124
+ | Three.js | Boo, the 3D ghost | three.js authors (mrdoob) | MIT | [GitHub](https://github.com/mrdoob/three.js) |
125
+ | ddgs | Web search for quick live facts | deedy5 | MIT | [GitHub](https://github.com/deedy5/ddgs) |
126
+ | wttr.in | Weather for quick live facts (data from WorldWeatherOnline) | Igor Chubin | Apache-2.0 | [GitHub](https://github.com/chubin/wttr.in) |
127
+ | NVIDIA PyTorch container | The Docker image's base (`nvcr.io/nvidia/pytorch:26.07-py3`) | NVIDIA | NVIDIA Deep Learning Container License | [NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch) |
128
+ | Qwen3.5-0.8B (earlier versions) | The first memory captioner | Qwen team, Alibaba Cloud | Apache-2.0 | [Hugging Face](https://huggingface.co/Qwen) |
129
+ | AEON-7 Nemotron 3 Nano Omni build (earlier versions) | The first camera model, from NVIDIA's Nemotron 3 Nano Omni | AEON-7 | NVIDIA Open Model License | [Hugging Face](https://huggingface.co/AEON-7) |
130
+
131
+ Full details: [CREDITS.md](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/blob/main/CREDITS.md)
media/promo-45s.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f1b0b65b3e1e34fe6e9c89cf6201f717d70fa9088dd61a77a6e924a5a89813b
3
+ size 4999521
media/promo-landscape.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c0fca1d697828ff053dfd7b695fb41613884cf8ff18ece9225efae6d62e3d9b8
3
+ size 6970480
media/promo-poster.jpg ADDED
media/ui-desktop-light.png ADDED

Git LFS Details

  • SHA256: c1d7834861db2b1ef5693589913451779991e3659e467298b828f095fb51bd1a
  • Pointer size: 131 Bytes
  • Size of remote file: 291 kB
media/ui-desktop.png ADDED

Git LFS Details

  • SHA256: 1f88a19f580d498d6170b893622af8d8196d6acd91ea7410fd1bbacc5272755d
  • Pointer size: 131 Bytes
  • Size of remote file: 300 kB
media/ui-hermes-card.png ADDED

Git LFS Details

  • SHA256: abc989bcffdc168a477ef9864b36a94ba205705a67286d09a445f44a6c6ec19f
  • Pointer size: 131 Bytes
  • Size of remote file: 110 kB
media/ui-phone-camera.png ADDED

Git LFS Details

  • SHA256: d2f1cc0511d5f91185f36b7fe6574bac833e85fc402a4090de324a2b2dddff73
  • Pointer size: 131 Bytes
  • Size of remote file: 464 kB