| --- |
| title: Praxis-Briefing |
| emoji: 🩺 |
| colorFrom: green |
| colorTo: blue |
| sdk: gradio |
| sdk_version: 5.50.0 |
| python_version: "3.12" |
| app_file: app.py |
| pinned: false |
| license: apache-2.0 |
| tags: |
| - build-small-hackathon |
| - backyard-ai |
| - tiny-titan |
| - off-brand |
| - best-demo |
| - well-tuned |
| - field-notes |
| - speech-to-speech |
| - german |
| - healthcare |
| - voice-assistant |
| - liquid-ai |
| - lfm2 |
| - fine-tuned |
| - gradio |
| - track:backyard |
| - achievement:welltuned |
| - achievement:offbrand |
| - achievement:fieldnotes |
| short_description: German speech-to-speech briefing for a GP, on LFM2.5-Audio |
| models: |
| - LiquidAI/LFM2.5-Audio-1.5B |
| - jempf/peitho-1.5b-v6 |
| datasets: |
| - jempf/peitho-v7-data |
| - wikimedia/wikipedia |
| --- |
| |
| # 🩺 Praxis-Briefing — a German voice for Liquid's LFM2.5-Audio |
|
|
| **Build Small Hackathon submission · Track: Backyard AI.** We taught Liquid AI's |
| [`LFM2.5-Audio-1.5B`](https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B) — a model |
| that ships **English-only** for speech-to-speech — to **speak and understand German**, |
| and wrapped it in a hands-free voice assistant for a solo physician's morning routine. |
|
|
| Ask Peitho a spoken question about today's (fictional) appointment schedule; it answers |
| out loud, in one short German sentence, in real time over WebRTC. |
|
|
| > **Build Small Hackathon** — Track: **🏡 Backyard AI** · Badges: **🐜 Tiny Titan**, |
| > **🎯 Well-Tuned**, **🎨 Off-Brand**, **📓 Field Notes**, **🎬 Best Demo** |
| > |
| > - 🎥 Demo video: **[Watch the demo on Vimeo](https://vimeo.com/1201527821)** |
| > - 📣 Social post: **[See the post on X](https://x.com/juliankempf99/status/2066589626991522303)** |
| > - 📓 Field notes write-up: [`FIELD_NOTES.md`](FIELD_NOTES.md) |
| > - 🧠 Model (1.5B, fine-tuned): [`jempf/peitho-1.5b-v6`](https://huggingface.co/jempf/peitho-1.5b-v6) |
|
|
| > ⚠️ **Not a medical device.** The schedule is mock data, there is no connection to any |
| > real practice management system, and answers are for demonstration only. |
|
|
| --- |
|
|
| ## Why this matters for a physician |
|
|
| A solo GP ("Hausärztin") starts every day context-switching between a screen and a patient. |
| The keyboard is a tax on attention: looking things up, typing notes, clicking through a |
| practice system — all while a patient is in the room. |
|
|
| **Voice is the natural interface for a clinician whose hands and eyes are busy.** A small, |
| fast, *on-device-class* speech-to-speech model means: |
|
|
| - **Hands-free, eyes-up.** Ask "Wer ist mein nächster Patient?" while washing hands or |
| walking to the next room — no screen, no keyboard. |
| - **Local language, local trust.** German in, German out. A US-English-only model is a |
| non-starter in a German practice; the fine-tune is the whole point. |
| - **Small enough to run privately.** At 1.5B parameters this class of model can run close |
| to the data — important when the domain is patient information. |
| - **Latency that fits a conversation.** Real-time speech-to-speech (not record → upload → |
| transcribe → LLM → TTS) keeps the interaction natural. |
|
|
| The mock "Tagesplan" use case is deliberately simple, but it stands in for the real prize: |
| a quiet, German-speaking assistant that surfaces the right fact at the right moment without |
| pulling a doctor out of the moment with their patient. |
|
|
| --- |
|
|
| ## What we actually built for the hackathon |
|
|
| The headline contribution is **a new language for an audio foundation model**: |
|
|
| 1. **A German speech-to-speech dataset pipeline** (`build_v7_dataset.py`) — synthetic, |
| reproducible, and resumable end-to-end. |
| 2. **A German fine-tune of LFM2.5-Audio-1.5B** (`jempf/peitho-1.5b-v6`) that produces |
| natural German audio output — the base model does not. |
| 3. **This real-time WebRTC demo** that makes the result tangible in a believable clinical |
| scenario. |
|
|
| ### The dataset: teaching the model to talk |
|
|
| `LFM2.5-Audio-1.5B` is documented as **English-only for speech-to-speech**. A control test |
| with the unmodified base model produced garbled, mixed-language text and robotic German |
| audio — confirming that **German capability has to be trained in, not prompted in.** |
|
|
| So we generated a synthetic German speech-to-speech corpus: |
|
|
| | Stage | Tooling | What happens | |
| |-------|---------|--------------| |
| | **1. Text** | Claude (Haiku + Sonnet) | ~10k German Q&A pairs across **8 intent buckets**: factual, smalltalk, casual, identity, instruction-following, refusal, and more. The **factual** bucket is seeded from **German Wikipedia** summaries so answers are grounded, not hallucinated. Numbers are spelled out for natural TTS. | |
| | **2. Speech** | ElevenLabs (`eleven_multilingual_v2`, 24 kHz PCM) | Each turn rendered to audio. **User turns rotate through 4 voices** (acoustic diversity, so the Conformer encoder generalizes to many real speakers); **assistant turns use a single consistent voice** — *this* is what Peitho sounds like. | |
| | **3. Upload** | 🤗 `huggingface_hub` | Packaged to a Hub dataset repo for training. | |
|
|
| Design choices that matter for a *speech* model: |
|
|
| - **Acoustic diversity on input, consistency on output.** Many voices teach robust listening; |
| one voice gives the model a stable identity to speak with. |
| - **Intent coverage, not just facts.** Greetings, identity, instruction-following and explicit |
| refusals teach the model *how to behave*, not only *what to know*. |
| - **Wikipedia-grounded factual turns** keep the German it learns true and broad. |
|
|
| ### The model |
|
|
| `jempf/peitho-1.5b-v6` is the LFM2.5-Audio-1.5B base with our German fine-tune weights |
| overlaid. Named after **Peitho**, the Greek goddess of persuasion and eloquence. |
|
|
| It is an early checkpoint: fluent, natural German speech-to-speech, but a 1.5B model is not |
| a reliable knowledge base. In this demo we lean on its real strength — German voice |
| interaction — and ground the actual facts via the in-app schedule and few-shot prompting. |
| Closing the grounding gap is exactly what the next dataset iteration targets. |
|
|
| --- |
|
|
| ## How the demo works |
|
|
| `app.py` loads stock `LFM2.5-Audio-1.5B` via `liquid_audio.demo.model`, then overlays the |
| v6 German weights from `jempf/peitho-1.5b-v6` with `accelerate.load_checkpoint_in_model`. |
| `chat.py` serves a real-time WebRTC speech-to-speech UI: |
|
|
| - **Push-to-talk** voice via [`fastrtc`](https://github.com/gradio-app/fastrtc) with VAD — |
| speak, pause, and Peitho replies with interleaved text + audio. |
| - The day's **Tagesplan** is injected into the conversation so answers are grounded in it, |
| with a few-shot pattern so the small model answers from the plan. |
| - Sampled decoding (`temp=0.2, topk=10`) for steady, repeatable answers. |
|
|
| ### Running it / configuration |
|
|
| WebRTC on Hugging Face Spaces needs a **TURN relay**. Because the free community TURN |
| servers (`turn.fastrtc.org`) are currently down |
| ([fastrtc#429](https://github.com/gradio-app/fastrtc/issues/429)), this Space uses |
| **Cloudflare Calls** TURN with your own free keys. Add two Space secrets: |
|
|
| | Secret | Where to get it | |
| |--------|-----------------| |
| | `CLOUDFLARE_TURN_KEY_ID` | Cloudflare dashboard → Calls → create a TURN App | |
| | `CLOUDFLARE_TURN_KEY_API_TOKEN` | same TURN App | |
|
|
| Without them the app still boots (STUN-only), but real-time mic relay may not connect. |
| Edit the mock schedule and prompts in `briefing.py`. |
|
|
| --- |
|
|
| ## How it maps to the prize pool |
|
|
| - **🏡 Backyard AI (track):** a practical tool for a real person we know — a solo GP. Voice is |
| the right interface for a clinician whose hands and eyes are on the patient, not a keyboard. |
| - **🐜 Tiny Titan:** the whole thing runs on a **1.5B** model — and not a text model, a full |
| **speech-to-speech** model doing real-time German audio in and audio out. Tiny weights, big lift. |
| - **🎯 Well-Tuned:** the German capability is a **fine-tune we published** to the Hub |
| ([`jempf/peitho-1.5b-v6`](https://huggingface.co/jempf/peitho-1.5b-v6)) — the base model is |
| English-only, so the fine-tune is the entire contribution. |
| - **🎨 Off-Brand:** a fully custom **EHR-style clinical interface** — bespoke light theme, an |
| electronic-health-record top bar, a tabular Sprechstundenplan, voice example chips, and a clean |
| answer panel. No default Gradio look left. |
| - **📓 Field Notes:** a written build report — see [`FIELD_NOTES.md`](FIELD_NOTES.md). |
| - **🎬 Best Demo:** a voice-first demo that films well — ask in German, hear German back, live. |
|
|
| --- |
|
|
| ## Stack |
|
|
| - **Model:** [`LiquidAI/LFM2.5-Audio-1.5B`](https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B) |
| + German fine-tune [`jempf/peitho-1.5b-v6`](https://huggingface.co/jempf/peitho-1.5b-v6) |
| - **Audio runtime:** [`liquid-audio`](https://github.com/Liquid4All/liquid-audio) (Mimi codec, interleaved text+audio) |
| - **Real-time transport:** `fastrtc` (WebRTC + VAD), Cloudflare TURN |
| - **UI:** Gradio 5 |
| - **Dataset pipeline:** Claude (text) + ElevenLabs (TTS) + German Wikipedia + 🤗 Hub |
|
|
