README.md

metadata

title: Praxis-Briefing
emoji: 🩺
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 5.50.0
python_version: '3.12'
app_file: app.py
pinned: false
license: apache-2.0
tags:
  - build-small-hackathon
  - backyard-ai
  - tiny-titan
  - off-brand
  - best-demo
  - well-tuned
  - field-notes
  - speech-to-speech
  - german
  - healthcare
  - voice-assistant
  - liquid-ai
  - lfm2
  - fine-tuned
  - gradio
  - track:backyard
  - achievement:welltuned
  - achievement:offbrand
  - achievement:fieldnotes
short_description: German speech-to-speech briefing for a GP, on LFM2.5-Audio
models:
  - LiquidAI/LFM2.5-Audio-1.5B
  - jempf/peitho-1.5b-v6
datasets:
  - jempf/peitho-v7-data
  - wikimedia/wikipedia

🩺 Praxis-Briefing — a German voice for Liquid's LFM2.5-Audio

Build Small Hackathon submission · Track: Backyard AI. We taught Liquid AI's LFM2.5-Audio-1.5B — a model that ships English-only for speech-to-speech — to speak and understand German, and wrapped it in a hands-free voice assistant for a solo physician's morning routine.

Ask Peitho a spoken question about today's (fictional) appointment schedule; it answers out loud, in one short German sentence, in real time over WebRTC.

Build Small Hackathon — Track: 🏡 Backyard AI · Badges: 🐜 Tiny Titan, 🎯 Well-Tuned, 🎨 Off-Brand, 📓 Field Notes, 🎬 Best Demo

• 🎥 Demo video: Watch the demo on Vimeo
• 📣 Social post: See the post on X
• 📓 Field notes write-up: FIELD_NOTES.md
• 🧠 Model (1.5B, fine-tuned): jempf/peitho-1.5b-v6

⚠️ Not a medical device. The schedule is mock data, there is no connection to any real practice management system, and answers are for demonstration only.

---

Why this matters for a physician

A solo GP ("Hausärztin") starts every day context-switching between a screen and a patient. The keyboard is a tax on attention: looking things up, typing notes, clicking through a practice system — all while a patient is in the room.

Voice is the natural interface for a clinician whose hands and eyes are busy. A small, fast, on-device-class speech-to-speech model means:

• Hands-free, eyes-up. Ask "Wer ist mein nächster Patient?" while washing hands or walking to the next room — no screen, no keyboard.
• Local language, local trust. German in, German out. A US-English-only model is a non-starter in a German practice; the fine-tune is the whole point.
• Small enough to run privately. At 1.5B parameters this class of model can run close to the data — important when the domain is patient information.
• Latency that fits a conversation. Real-time speech-to-speech (not record → upload → transcribe → LLM → TTS) keeps the interaction natural.

The mock "Tagesplan" use case is deliberately simple, but it stands in for the real prize: a quiet, German-speaking assistant that surfaces the right fact at the right moment without pulling a doctor out of the moment with their patient.

---

What we actually built for the hackathon

The headline contribution is a new language for an audio foundation model:

1. A German speech-to-speech dataset pipeline (build_v7_dataset.py) — synthetic, reproducible, and resumable end-to-end.
2. A German fine-tune of LFM2.5-Audio-1.5B (jempf/peitho-1.5b-v6) that produces natural German audio output — the base model does not.
3. This real-time WebRTC demo that makes the result tangible in a believable clinical scenario.

The dataset: teaching the model to talk

LFM2.5-Audio-1.5B is documented as English-only for speech-to-speech. A control test with the unmodified base model produced garbled, mixed-language text and robotic German audio — confirming that German capability has to be trained in, not prompted in.

So we generated a synthetic German speech-to-speech corpus:

Stage	Tooling	What happens
1. Text	Claude (Haiku + Sonnet)	~10k German Q&A pairs across 8 intent buckets: factual, smalltalk, casual, identity, instruction-following, refusal, and more. The factual bucket is seeded from German Wikipedia summaries so answers are grounded, not hallucinated. Numbers are spelled out for natural TTS.
2. Speech	ElevenLabs (eleven_multilingual_v2, 24 kHz PCM)	Each turn rendered to audio. User turns rotate through 4 voices (acoustic diversity, so the Conformer encoder generalizes to many real speakers); assistant turns use a single consistent voice — this is what Peitho sounds like.
3. Upload	🤗 huggingface_hub	Packaged to a Hub dataset repo for training.

Design choices that matter for a speech model:

• Acoustic diversity on input, consistency on output. Many voices teach robust listening; one voice gives the model a stable identity to speak with.
• Intent coverage, not just facts. Greetings, identity, instruction-following and explicit refusals teach the model how to behave, not only what to know.
• Wikipedia-grounded factual turns keep the German it learns true and broad.

The model

jempf/peitho-1.5b-v6 is the LFM2.5-Audio-1.5B base with our German fine-tune weights overlaid. Named after Peitho, the Greek goddess of persuasion and eloquence.

It is an early checkpoint: fluent, natural German speech-to-speech, but a 1.5B model is not a reliable knowledge base. In this demo we lean on its real strength — German voice interaction — and ground the actual facts via the in-app schedule and few-shot prompting. Closing the grounding gap is exactly what the next dataset iteration targets.

---

How the demo works

app.py loads stock LFM2.5-Audio-1.5B via liquid_audio.demo.model, then overlays the v6 German weights from jempf/peitho-1.5b-v6 with accelerate.load_checkpoint_in_model. chat.py serves a real-time WebRTC speech-to-speech UI:

• Push-to-talk voice via fastrtc with VAD — speak, pause, and Peitho replies with interleaved text + audio.
• The day's Tagesplan is injected into the conversation so answers are grounded in it, with a few-shot pattern so the small model answers from the plan.
• Sampled decoding (temp=0.2, topk=10) for steady, repeatable answers.

Running it / configuration

WebRTC on Hugging Face Spaces needs a TURN relay. Because the free community TURN servers (turn.fastrtc.org) are currently down (fastrtc#429), this Space uses Cloudflare Calls TURN with your own free keys. Add two Space secrets:

Secret	Where to get it
CLOUDFLARE_TURN_KEY_ID	Cloudflare dashboard → Calls → create a TURN App
CLOUDFLARE_TURN_KEY_API_TOKEN	same TURN App

Without them the app still boots (STUN-only), but real-time mic relay may not connect. Edit the mock schedule and prompts in briefing.py.

---

How it maps to the prize pool

• 🏡 Backyard AI (track): a practical tool for a real person we know — a solo GP. Voice is the right interface for a clinician whose hands and eyes are on the patient, not a keyboard.
• 🐜 Tiny Titan: the whole thing runs on a 1.5B model — and not a text model, a full speech-to-speech model doing real-time German audio in and audio out. Tiny weights, big lift.
• 🎯 Well-Tuned: the German capability is a fine-tune we published to the Hub (jempf/peitho-1.5b-v6) — the base model is English-only, so the fine-tune is the entire contribution.
• 🎨 Off-Brand: a fully custom EHR-style clinical interface — bespoke light theme, an electronic-health-record top bar, a tabular Sprechstundenplan, voice example chips, and a clean answer panel. No default Gradio look left.
• 📓 Field Notes: a written build report — see FIELD_NOTES.md.
• 🎬 Best Demo: a voice-first demo that films well — ask in German, hear German back, live.

---

Stack

• Model: LiquidAI/LFM2.5-Audio-1.5B
  • German fine-tune jempf/peitho-1.5b-v6
• Audio runtime: liquid-audio (Mimi codec, interleaved text+audio)
• Real-time transport: fastrtc (WebRTC + VAD), Cloudflare TURN
• UI: Gradio 5
• Dataset pipeline: Claude (text) + ElevenLabs (TTS) + German Wikipedia + 🤗 Hub