Spaces:
Sleeping
Sleeping
Reachy DM landing page: write-up + tags + video placeholder
Browse files- README.md +44 -8
- app.py +96 -0
- requirements.txt +1 -0
README.md
CHANGED
|
@@ -1,13 +1,49 @@
|
|
| 1 |
---
|
| 2 |
-
title: Reachy
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
-
sdk_version: 6.18.0
|
| 8 |
-
python_version: '3.13'
|
| 9 |
app_file: app.py
|
| 10 |
-
pinned:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Reachy DM — Thousand Token Wood
|
| 3 |
+
emoji: 🎲
|
| 4 |
+
colorFrom: purple
|
| 5 |
+
colorTo: indigo
|
| 6 |
sdk: gradio
|
|
|
|
|
|
|
| 7 |
app_file: app.py
|
| 8 |
+
pinned: true
|
| 9 |
+
short_description: Whimsical all-Qwen voice DM, embodied by Reachy Mini
|
| 10 |
+
tags:
|
| 11 |
+
- track:wood
|
| 12 |
+
- sponsor:modal
|
| 13 |
+
- achievement:bestdemo
|
| 14 |
+
- achievement:bestagent
|
| 15 |
+
- achievement:offbrand
|
| 16 |
+
- agent
|
| 17 |
+
- voice
|
| 18 |
+
- mcp
|
| 19 |
+
- tabletop-rpg
|
| 20 |
+
- qwen
|
| 21 |
+
- reachy_mini
|
| 22 |
---
|
| 23 |
|
| 24 |
+
# 🎲 Reachy DM — Thousand Token Wood
|
| 25 |
+
|
| 26 |
+
A whimsical AI tabletop **dungeon master you play out loud**, embodied by a **Reachy Mini** robot.
|
| 27 |
+
It narrates scenes, voices NPCs in distinct designed voices, rolls dice, reads your physical
|
| 28 |
+
character sheet through a camera, sets the mood with smart lights, and remembers your party —
|
| 29 |
+
an **all-Qwen, open-weights** stack with every model under the 32B cap.
|
| 30 |
+
|
| 31 |
+
The full embodied experience (robot + character voices + dice/sheet widgets + story-reactive
|
| 32 |
+
LIFX lighting) is a **local hardware rig**, so this Space is the showcase: the **demo video** and
|
| 33 |
+
a write-up of how it was built. See the page for details.
|
| 34 |
+
|
| 35 |
+
- **DM brain:** Qwen3-30B-A3B-Instruct-2507 (FP8) on **Modal** (H100)
|
| 36 |
+
- **Character voices:** Qwen3-TTS-12Hz-1.7B on **Modal** (L4)
|
| 37 |
+
- **Vision ("read the table"):** Qwen3-VL-8B on **Modal** (L40S)
|
| 38 |
+
- **Tools via MCP:** dice / character sheet / player choice (rendered as Pip-Boy widgets),
|
| 39 |
+
per-character voices, durable memory, camera, robot motion, Home Assistant lighting.
|
| 40 |
+
|
| 41 |
+
**Track:** Thousand Token Wood (whimsical). **Code:**
|
| 42 |
+
[github.com/olaservo/reachy_mini_conversation_app](https://github.com/olaservo/reachy_mini_conversation_app)
|
| 43 |
+
(branch `cascade-integration`).
|
| 44 |
+
|
| 45 |
+
## Disclaimer
|
| 46 |
+
A personal, non-commercial creative project for the Build Small hackathon, running *Fallout: The
|
| 47 |
+
Roleplaying Game* (Modiphius 2d20) from a copy I own. **Not affiliated with, endorsed by, or
|
| 48 |
+
sponsored by Bethesda Softworks or Modiphius Entertainment.** All rights to those properties
|
| 49 |
+
belong to their respective owners.
|
app.py
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Reachy DM — Build Small hackathon submission landing page.
|
| 2 |
+
|
| 3 |
+
A whimsical all-Qwen voice dungeon master you play out loud, embodied by a Reachy Mini.
|
| 4 |
+
The full experience (robot + character voices + dice/sheet widgets + story-reactive lights)
|
| 5 |
+
is a local hardware rig — this Space is the showcase: the demo video + how it was built.
|
| 6 |
+
"""
|
| 7 |
+
|
| 8 |
+
import gradio as gr
|
| 9 |
+
|
| 10 |
+
# --- Fill these in after recording / posting ---
|
| 11 |
+
DEMO_VIDEO_EMBED_URL = "" # e.g. "https://www.youtube.com/embed/XXXXXXXXXXX"
|
| 12 |
+
SOCIAL_POST_URL = "" # link to the social post
|
| 13 |
+
GITHUB_URL = "https://github.com/olaservo/reachy_mini_conversation_app"
|
| 14 |
+
|
| 15 |
+
PIPBOY_CSS = """
|
| 16 |
+
.gradio-container { background: radial-gradient(circle at 50% 0%, #0b2a16 0%, #05140a 80%) !important; }
|
| 17 |
+
#pip * { color: #7CFC8A !important; font-family: 'Courier New', monospace !important; }
|
| 18 |
+
#pip h1 { text-shadow: 0 0 10px #2f8; letter-spacing: 1px; }
|
| 19 |
+
#pip a { color: #b6ff8a !important; text-decoration: underline; }
|
| 20 |
+
#pip table { border-color: #2f8a4a !important; }
|
| 21 |
+
#pip code { background: rgba(47,138,74,0.12) !important; border: 1px solid #2f8a4a; padding: 1px 4px; }
|
| 22 |
+
#pip .vault { border: 1px solid #2f8a4a; border-radius: 10px; padding: 14px 18px; background: rgba(47,138,74,0.06); }
|
| 23 |
+
"""
|
| 24 |
+
|
| 25 |
+
VIDEO_BLOCK = (
|
| 26 |
+
f'<div class="vault"><iframe width="100%" height="420" src="{DEMO_VIDEO_EMBED_URL}" '
|
| 27 |
+
'frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; '
|
| 28 |
+
'gyroscope; picture-in-picture" allowfullscreen></iframe></div>'
|
| 29 |
+
if DEMO_VIDEO_EMBED_URL
|
| 30 |
+
else '<div class="vault">▶ <b>Demo video coming soon.</b> Check back shortly.</div>'
|
| 31 |
+
)
|
| 32 |
+
|
| 33 |
+
SOCIAL_LINE = f"**Social post:** [link]({SOCIAL_POST_URL}) · " if SOCIAL_POST_URL else ""
|
| 34 |
+
|
| 35 |
+
WRITEUP = f"""
|
| 36 |
+
# 🎲 Reachy DM — a whimsical AI tabletop dungeon master you play *out loud*
|
| 37 |
+
|
| 38 |
+
**Track: Thousand Token Wood (whimsical).** Reachy DM is an AI Game Master for a Fallout
|
| 39 |
+
tabletop RPG (Modiphius 2d20). You talk to it; it narrates vivid scenes, voices the NPCs in
|
| 40 |
+
distinct designed voices, rolls the dice, reads your physical character sheet through a camera,
|
| 41 |
+
sets the mood with your room's smart lights, and remembers your party — all embodied by a
|
| 42 |
+
**Reachy Mini** robot. Every model in the stack is open-weights and under the 32B cap.
|
| 43 |
+
|
| 44 |
+
{SOCIAL_LINE}**Code:** [{GITHUB_URL.split('//')[-1]}]({GITHUB_URL}) (branch `cascade-integration`)
|
| 45 |
+
|
| 46 |
+
> The full experience is a **local hardware rig** (robot · smart lights · companion screen),
|
| 47 |
+
> so the **demo video above** is the showcase. This page explains how it works.
|
| 48 |
+
|
| 49 |
+
## How it works — an all-Qwen cascade
|
| 50 |
+
A voice loop (Whisper STT · Silero VAD) drives a **Qwen brain** that orchestrates tools and
|
| 51 |
+
speaks back in designed character voices, reading the table with vision when you show it something.
|
| 52 |
+
|
| 53 |
+
| Stage | Model | Where |
|
| 54 |
+
|---|---|---|
|
| 55 |
+
| Speech-to-text | Whisper | OpenAI API |
|
| 56 |
+
| Voice activity | Silero VAD | local (CPU) |
|
| 57 |
+
| **DM brain (logic + tool-calling)** | **Qwen3-30B-A3B-Instruct-2507 (FP8)** | **Modal** (1×H100) |
|
| 58 |
+
| **Character voices (11 designed)** | **Qwen3-TTS-12Hz-1.7B** | **Modal** (L4) |
|
| 59 |
+
| **Vision ("read the table")** | **Qwen3-VL-8B-Instruct** | **Modal** (L40S) |
|
| 60 |
+
|
| 61 |
+
The brain calls tools over **MCP**: dice / character sheet / player choice render as live
|
| 62 |
+
**Pip-Boy widgets** (an MCP-Apps custom UI), plus per-character voices (`speak_as`), durable
|
| 63 |
+
**memory**, the camera, robot motion/expression, and Home Assistant smart lighting.
|
| 64 |
+
|
| 65 |
+
### Best Use of Modal
|
| 66 |
+
The three GPU models — **brain (H100)**, **character-voice TTS (L4)**, **vision (L40S)** — are all
|
| 67 |
+
served on **Modal** (serverless vLLM + a custom FastAPI TTS server). Modal is the runtime
|
| 68 |
+
compute backbone of the whole experience.
|
| 69 |
+
|
| 70 |
+
### Badges
|
| 71 |
+
- **Best Agent** — a genuine multi-step agent choosing among dice, sheets, choices, voices,
|
| 72 |
+
vision, memory, lighting, and robot motion, all under the 32B cap.
|
| 73 |
+
- **Best Demo** — a robot Game Master that voices NPCs, rolls dice on screen, dims your lights
|
| 74 |
+
for combat, and reads your printed character sheet with a camera.
|
| 75 |
+
- **Off Brand** — tool calls render as a custom **Pip-Boy** UI well past the default Gradio look.
|
| 76 |
+
|
| 77 |
+
## Models & constraints
|
| 78 |
+
All-Qwen, open-weights, **each model < 32B total** (30B brain · 8B vision · 1.7B TTS), plus
|
| 79 |
+
Whisper/Silero for speech. Built on a fork of the Pollen Robotics Reachy Mini conversation app.
|
| 80 |
+
|
| 81 |
+
## Disclaimer
|
| 82 |
+
Reachy DM is a personal, non-commercial creative project built for the Build Small hackathon. It
|
| 83 |
+
runs *Fallout: The Roleplaying Game* (Modiphius 2d20 system) using rules and content from a copy
|
| 84 |
+
I own, as a fan would at their own table. It is **not affiliated with, endorsed by, or sponsored
|
| 85 |
+
by Bethesda Softworks or Modiphius Entertainment**. *Fallout* is a trademark of Bethesda Softworks;
|
| 86 |
+
*Fallout: The Roleplaying Game* is published by Modiphius Entertainment. All rights to those
|
| 87 |
+
properties belong to their respective owners.
|
| 88 |
+
"""
|
| 89 |
+
|
| 90 |
+
with gr.Blocks(css=PIPBOY_CSS, title="Reachy DM — Thousand Token Wood") as demo:
|
| 91 |
+
with gr.Column(elem_id="pip"):
|
| 92 |
+
gr.HTML(VIDEO_BLOCK)
|
| 93 |
+
gr.Markdown(WRITEUP)
|
| 94 |
+
|
| 95 |
+
if __name__ == "__main__":
|
| 96 |
+
demo.launch()
|
requirements.txt
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
gradio>=5,<6
|