README.md

metadata

title: Paw Talk
emoji: 🐾
colorFrom: red
colorTo: yellow
sdk: gradio
sdk_version: 6.18.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: A pet's snarky inner monologue, voiced to match the animal.
tags:
  - track:wood
  - sponsor:openbmb
  - sponsor:openai

🐾 Paw Talk — give your pet its inner monologue

Upload a short pet video; the app watches it and dubs the animal's snarky inner monologue in a voice that fits the animal, in the character style you pick.

Built for the Build Small Hackathon · targeting the OpenBMB MiniCPM and OpenAI Codex sponsor prizes.

How it works (MiniCPM + VoxCPM2, on free HF ZeroGPU)

1. MiniCPM-V 4.6 watches sampled frames → an objective description of the animal & scene.
2. MiniCPM-V 4.6 (text-only) → infers a fitting voice timbre from the species/size, then rewrites the scene into a first-person inner monologue in character (thoughts, not narration).
3. VoxCPM2 speaks it; a per-animal pitch/formant shift pushes the voice out of the "human" range so it reads as a small creature.
4. ffmpeg dubs it onto the clip with time-synced burned-in subtitles; the voice is stretched to cover the whole clip (≤30s).

No model is over 32B; nothing is "right/wrong", so the small models can't look dumb — they just perform.

Demo & links

• 🎬 Demo video: https://huggingface.co/spaces/build-small-hackathon/paw-talk/resolve/main/demo.mp4
• 🐦 Social post: https://x.com/lnghul413432/status/2066548599224422470

Architecture

MiniCPM-V and VoxCPM2 run in-Space on HF ZeroGPU — the GPU is requested only inside the @spaces.GPU call for each request and released right after, so it is free and self-contained (no external service, no secrets). Frame sampling, subtitle rendering and ffmpeg muxing run on CPU.