README.md

---
title: ChalkTalk
emoji: 🎓
colorFrom: indigo
colorTo: purple
sdk: gradio
app_file: app.py
pinned: false
short_description: An AI tutor that talks and draws, live.
tags:
  - track:backyard
  - sponsor:openai
  - sponsor:nvidia
  - sponsor:modal
  - achievement:offgrid
  - achievement:welltuned
  - achievement:offbrand
---

# 🎓 ChalkTalk

**The AI tutor that talks *and* draws — live, at the same time.**

You type a topic and a tutor explains it out loud while a whiteboard fills in beside it. Every labeled part lights up the exact moment the tutor says its name.

## Watch the demo

<video controls src="https://huggingface.co/datasets/squaredcuber/chalktalk-demo/resolve/main/chalktalk-demo.mp4"></video>

Direct link: [https://huggingface.co/datasets/squaredcuber/chalktalk-demo/resolve/main/chalktalk-demo.mp4](https://huggingface.co/datasets/squaredcuber/chalktalk-demo/resolve/main/chalktalk-demo.mp4)

▶ **Live demo:** [https://belong-picking-platinum-represents.trycloudflare.com](https://belong-picking-platinum-represents.trycloudflare.com)

## How it works

- **PersonaPlex** — a full-duplex, Moshi-architecture 7B speech model that listens and speaks at the same time, so you can interrupt and ask questions mid-explanation.
- **Nemotron SVG director** — an NVIDIA Nemotron model that draws a clean, labeled SVG diagram for each part of the lesson, streamed to a live whiteboard.
- **Synced reveal** — the tutor inner-monologue drives the board: each labeled element appears the instant its keyword is spoken, with synonym-aware matching so paraphrases still land.
- **Auto-progressing lessons** — it teaches one part deeply, then voice and visuals advance together, over one persistent connection.
- **Runs locally on a single GPU**, wrapped in a shader-animated Gradio app.

## Credits

Built by **Codex**.
Inference & Finetuning on **Modal**