Spaces:
Running
Download README.md from joemartis/Video2Guide: direct link, hf CLI and curl.
- Browser
- Download file 8.54 kB
-
https://huggingface.co/spaces/joemartis/Video2Guide/resolve/main/README.md
- Command line
-
hf download hf://spaces/joemartis/Video2Guide/README.md
-
curl -L -o README.md https://huggingface.co/spaces/joemartis/Video2Guide/resolve/main/README.md
title: The Describatron
emoji: π€
colorFrom: indigo
colorTo: purple
sdk: docker
app_port: 7860
pinned: false
The Describatron
Generate WCAG 2.2 AA-ready HTML study guides from instructional videos β or from a set of slide images.
Takes a lecture plus a transcript (VTT/SRT) and produces a flowing, accessible HTML document: visual content shown as images, narration aligned to each segment, on-screen text captured, instructor talking-head footage filtered out, and optional Claude-powered enrichment (scene descriptions, alt text, on-screen-text cleanup, LaTeX math, section titles).
Two ways in
- From a video (
/) β upload a video + transcript; the app detects slide changes, extracts a representative frame per scene, and filters out talking-head shots. - From slide images (
/images) β already have slide exports? Upload the images plus (optionally) a transcript. Name each image with anHH-MM-SStimestamp (e.g.00-05-30.png) to anchor it exactly; if the names have no timestamps, the AI reads the transcript and places each slide where it's discussed (or, with no API key, spreads them evenly). No transcript at all β a slides-only guide ordered by filename. No video upload, no scene detection.
Both paths feed the same downstream pipeline and editor.
What it does (video path)
- Scene preview β see extracted frames before committing; deselect any you don't need (duplicates, transitions, stray shots).
- Scene detection β finds moments where visual content shifts (new slide, diagram, animation step). Frame extraction is deterministic (frame-accurate seeking).
- Instructor filtering β drops frames that are mostly a talking head (face detection). Tune the threshold in the sidebar.
- On-screen text β Tesseract OCR, or Claude reads the slide directly.
- Transcript alignment β maps narration to each visual segment by timestamp.
- LLM enrichment (optional, requires Anthropic API key) β Claude reads each frame and can produce: a scene description (an open paragraph under the slide that doubles as the long-form image description), WCAG-grade alt text, cleaned on-screen text, LaTeX math equations, and section titles. It can also segment the transcript into topics across slides.
- HTML generation β an accessible document with proper landmarks, headings,
lang, skip links, collapsible transcript, and MathJax-rendered equations.
The editor and outputs
Choosing Editor output opens an in-browser editor to refine everything before publishing. It supports:
- Editing titles, scene descriptions, alt text, transcript, on-screen text, and math.
- Per-element include toggles (and document-wide bulk toggles) β choose exactly what carries into the published guide, per section: scene description, alt-text, on-screen text, transcript, audio.
- Save edits (JSON) β a self-contained file that embeds the slides. Anyone can
reopen it at
/editand keep editing β no images to ship, no re-run. - Download as a single self-contained HTML file, semantic Markdown, or a
ZIP bundle (HTML +
static/assets).
Accessibility
This tool produces a WCAG 2.2 AA-ready scaffold, not a certified-compliant
document. Auto-generated descriptions/alt-text are drafts flagged for human review
in the editor. Heading hierarchy, landmarks, lang, skip links, and reading order
are generated correctly out of the box.
The intended pattern is the scene description: an open paragraph under each
slide that both sighted and screen-reader users read in the page body. When a
description is present, the image's alt is emptied (decorative) so the meaning
isn't announced twice. The editor lets you confirm or rewrite each one, and choose
per-section what carries into the published guide. See DATA_HANDLING.md for what
data is sent where when the AI features are on.
Quick start
Requires Python 3.10β3.13 and system packages ffmpeg + tesseract-ocr.
pip install -r requirements.txt
uvicorn app.main:app --reload
Then open http://localhost:8000.
Environment variables
| Variable | Default | Description |
|---|---|---|
MAX_VIDEO_MB |
500 |
Video upload size cap in MB. Set to 0 to disable. |
MAX_TRANSCRIPT_MB |
5 |
Transcript upload size cap in MB. Set to 0 to disable. |
MAX_IMAGE_MB |
25 |
Per-image upload cap (images mode). Set to 0 to disable. |
MAX_EDIT_MB |
150 |
Cap for a self-contained edits JSON uploaded to /edit. |
ANTHROPIC_API_KEY |
(unset) | Enables LLM enrichment features. Can also be entered per-job in the UI. |
ANTHROPIC_BASE_URL |
(unset) | Route Anthropic requests to a custom compatible endpoint. Optional; see below. |
PARLEY_BASE_URL |
https://parley.api.mit.edu |
Endpoint used for MIT Parley keys. |
ANTHROPIC_MODEL |
claude-haiku-4-5-20251001 |
Model ID for all LLM calls. Set this if your endpoint names models differently β it must be vision-capable or slide analysis returns blank/hallucinated descriptions. |
API keys and MIT Parley
You can paste either a normal Anthropic key (sk-ant-β¦) or an MIT Parley
key (sk-parley-β¦) into the API-key field (or set it as ANTHROPIC_API_KEY).
Parley keys are automatically routed to the Parley proxy (PARLEY_BASE_URL);
Anthropic keys go to Anthropic. No configuration needed β the routing is by key
prefix. To point normal keys at some other Anthropic-compatible proxy, set
ANTHROPIC_BASE_URL (this never overrides Parley-key routing).
# Example: no upload limits, LLM features on
MAX_VIDEO_MB=0 ANTHROPIC_API_KEY=sk-ant-... uvicorn app.main:app --reload
Project layout
app/
main.py FastAPI app: / (video), /images, /edit, job queue, preview + asset endpoints
jobs.py In-memory job registry (SSE progress, TTL, quota)
pipeline/
orchestrator.py Pipeline entry point (run_pipeline); video_path optional
transcript.py VTT/SRT parsing (incl. hourless timestamps) + segment alignment
scenes.py Scene detection + frame-accurate extraction
image_input.py Images mode: HH-MM-SS filename parsing β SceneFrame list
filter.py Instructor-frame filtering (Haar + YuNet)
ocr.py Tesseract OCR
llm_extract.py Claude extraction (scene description, alt text, OCR, math, titles, topics)
html_gen.py WCAG HTML rendering (guide, single, review) + metadata round-trip + bundling
metadata.py Shared Pydantic schema (segments, include flags, inlined assets)
templates/
index.html Video upload form + scene picker UI
images.html Images-mode upload page
edit.html Open a shared self-contained edits JSON
guide.html Published study guide template
review.html In-browser editor template
_doc_styles.html Shared CSS for guide and review outputs
tests/
Deploy to Hugging Face Spaces
This repo ships a Dockerfile compatible with Hugging Face Spaces (free CPU tier).
- Create a new Space β pick Docker as the SDK and Blank as the template.
- Push this repo to the Space's git remote:
git remote add space https://huggingface.co/spaces/<your-user>/<space-name> git push space <branch>:main - Spaces reads the YAML frontmatter at the top of this README, builds the Dockerfile, and exposes the app on port
7860.
Notes:
- Uploads land in
/tmpinside the container (ephemeral β one-shot processing per visitor). - The free CPU tier handles a 20-minute lecture in a few minutes.
- The Space sleeps after inactivity and wakes on the next request (~30 s cold start).
- Do not set
X-Frame-Optionsor CSP headers at the app level β HF Spaces proxies the app through its own infrastructure and those headers will break the Space. Apply them at the reverse-proxy / CDN layer instead.
Status
Beta. The pipeline runs end-to-end on common inputs. Known gaps:
- A transcript file is required (there's an optional Whisper extra:
pip install -e .[asr]). - OCR is English-only by default; Claude-based on-screen text handles other cases better.
- Face detection uses OpenCV Haar cascades + YuNet (fast, good enough for talking heads).
- The
/editshared editor has no alt-frame switching or live re-OCR (those need a live job); it edits all text, descriptions, toggles, and math.