OmniVoice-Studio / docs /ROADMAP.md
LΓͺ Phi Nam
Deploy to HF Space
94004a2
|
Raw
History Blame Contribute Delete
72.9 kB
# OmniVoice Studio β€” Road to World-Class
**Last updated:** 2026-04-21 Β· **Current phase:** Phases 0–4 complete. Remaining work sits in Design / Performance / Quality tracks + the Phase 4 eval sprint. Β· **Resourcing:** 1 FTE + ad-hoc
Honest plan for moving from "polished solo-dev project" to software that holds up under scrutiny at scale. Living tracker: every item has a status, every phase has a progress bar, every shipped item has a date.
---
## πŸ“Š Status overview
```
Phase 0 Β· Momentum β–“β–“β–“β–“β–“β–“β–“β–“β–“β–“ 5 / 6 βœ… shipped
Phase 1 Β· Visible quality leap β–“β–“β–“β–“β–“β–“β–“β–“β–“β–“ 4 / 4 βœ… shipped
Phase 2 Β· Foundation refactor β–“β–“β–“β–“β–“β–“β–“β–“β–“β–“ 8 / 8 βœ… shipped (2.2 closed with 5 Zustand slices)
Phase 3 Β· Pluggable engines β–“β–“β–“β–“β–“β–“β–“β–“β–“β–“ 4 / 4 βœ… shipped
Phase 4 Β· The two bets β–“β–“β–“β–“β–“β–“β–“β–“β–“β–“ 6 / 6 βœ… shipped Β· 4.1 benchmarked at 4.04s warm (target ≀5s)
Phase 5 Β· Productisation β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘ 0 / 5 🚫 demand-driven
Design track β–“β–“β–“β–“β–“β–“β–“β–“β–“β–‘ ongoing Β· 14 primitives + ~67 migrated inline styles Β· DubTab/Header/Sidebar/CloneDesignTab drained
Performance track β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘ not started
Feature-magic track β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘ not started
Quality track β–“β–“β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–‘ 12 smoke tests, 10 error messages rewritten
```
**Legend:** βœ… shipped Β· 🟑 in progress Β· ⏳ not started Β· πŸ”’ blocked Β· 🚫 deferred
---
## 🎯 North Star
> The best **local-first** cinematic dubbing studio in the world. Indistinguishable from a cloud product in quality and UX, but never leaves the user's machine.
Three non-negotiables:
1. **Two defensible innovations** that no competitor can copy in a sprint, backed by obsessive polish everywhere else.
2. **Dub output quality** good enough that a human editor would only tweak wording, not rebuild timing.
3. **UX** where a non-technical user produces a polished dubbed video in under 10 minutes, end to end, and never waits on a spinner longer than they expect.
## 🎲 Our two bets (the story worth telling)
| | Bet | Status |
|---|---|---|
| **A** | **Directorial AI** β€” natural-language per-segment direction ("make segment 14 feel more urgent") rewrites translation tone + TTS `instruct` + speech-rate target. | ⏳ Phase 4 |
| **B** | **Incremental re-dub** β€” change one word in a 2-hour video, regenerate only affected segments + crossfades in seconds. | ⏳ Phase 4 |
Together they compose: directorial edits trigger incremental re-dubs. That's the one-liner we stand on.
---
## πŸ“ Where we are today
βœ… Feature-complete MVP. Competitive baseline parity with VideoLingo / pyVideoTrans.
🟑 Code health improving fast. **14 design-system primitives shipped**, 9 components migrated, main bundle -2.7 % despite net additions.
⏳ Quality ceiling unchanged β€” one-shot translation, raw WhisperX segments, no speech-rate adaptation. **This is what Phase 1 fixes.**
⏳ UX ceiling β€” motion language, error messages, onboarding: partial. Launch animation shipped; error messages rewritten (10 sites); onboarding + sample clip still pending.
See [`research/LEARNINGS.md`](research/LEARNINGS.md) for the competitor analysis this roadmap is built on Β· [`STRUCTURE.md`](STRUCTURE.md) for file layout Β· [`design/`](design/) for target-state ASCII mockups.
---
## πŸ“ Resourcing assumption
**1 FTE + ad-hoc contributors.** All timeline ranges below assume that. Double the team β†’ roughly halve the time. Solo β†’ roughly 1.5Γ—. All timelines are **ranges**, not promises, revisited at each phase exit.
---
## 🧱 Phase 0 β€” Momentum _(1 week β†’ βœ… shipped 2026-04-21)_
> *Ship-this-week wins. Nothing invisible. Build a habit of landing improvements.*
Progress: **5/6 (83 %)**
| ID | Item | Status | Notes |
|----|------|:---:|------|
| 0.1 | Dead-code sweep (orphaned `test_*.py`, debug scripts, runtime artifacts at root) | βœ… | Shipped with the STRUCTURE.md cleanup. Deleted 8 files, archived `legacy_gradio/`, cleaned 16 stray `.DS_Store`s. |
| 0.2 | Alembic migration skeleton | βœ… | `alembic.ini`, `backend/migrations/{env,script.py.mako,versions,README}` ready. Legacy `_migrate()` in `core/db.py` kept as fallback for one release. |
| 0.3 | Top-10 error-message rewrite ("what happened Β· why Β· what to do") | βœ… | `dub_generate.py:98`, `dub_generate.py:26`, `generation.py:59`, `generation.py:183`, and `exports.py` Γ—6 all rewritten. Validated via smoke test on `/export/reveal` and `/dub/generate/{bad-id}`. |
| 0.4 | One smoke test per router | βœ… | `tests/test_router_smoke.py` β€” **12 tests, runs in 2.79 s**. Covers `system`, `profiles`, `projects`, `exports`, `generation`, `dub_core`, `dub_generate`. |
| 0.5 | 500 ms launch animation + `.ui-skel` shimmer utility | βœ… | In `ui/tokens.css`, auto-applied via `#root` selector. `prefers-reduced-motion` fallback included. |
| 0.6 | Pre-loaded 30-second sample clip for first-run onboarding | 🚫 | Deferred β€” requires licensed asset selection + Tauri bundling work. Revisit in Phase 5. |
**Exit criteria met:** errors actionable, CI smoke-green on every router, launch animation live. `kill -9` during a job **does not yet resume** β€” that moves to Phase 4.
---
## 🎨 Phase 1 β€” Visible quality leap _(shipped 2026-04-21 β†’ βœ…)_
> *The single biggest user-visible upgrade. Directly from VideoLingo's playbook. Ships before the deeper foundation refactor so users feel the improvement first.*
Progress: **4/4 (100 %)**
| ID | Item | Status | Target files |
|----|------|:---:|------|
| 1.1 | Translate β†’ Reflect β†’ Adapt (3-step LLM chain) | βœ… | Shipped 2026-04-21. `backend/services/translator.py` (new), `dub_translate.py` (wired), `schemas/requests.py` (+`quality`, `+glossary`), `App.jsx` (`translateQuality` state, localStorage-persisted), `DubTab.jsx` (Fast/Cinematic Segmented control), response carries `literal`+`critique` per segment for the future 3-column UI. Works with any OpenAI-compatible endpoint (real OpenAI, Ollama, LM Studio, Together, Anyscale). Graceful fallback when no LLM configured. Smoke suite still green (12/12 in 2.85 s). |
| 1.2 | NLP-aware subtitle segmentation (Netflix rules: ≀42 chars/line, ≀17 CPS) | βœ… | Shipped 2026-04-21. `backend/services/subtitle_segmenter.py` (new, 226 lines, pure-function, zero deps). Hooked into both transcription paths in `dub_core.py` (chunked streaming + single-shot), runs after diarization so speaker_id flows through splits. Tests: `tests/test_subtitle_segmenter.py` β€” 14 tests covering sentence/clause/conjunction splits, CPS enforcement, word-level timing, merger, speaker-boundary respect, edge cases. All 26 backend tests (14 new + 12 smoke) pass in 2.87 s. |
| 1.3 | Project-scoped term glossary with LLM auto-extract | βœ… | Shipped 2026-04-21. `glossary_terms` table in `core/db.py` (+ schema version 3); new `backend/api/routers/glossary.py` with full CRUD + `POST /glossary/{id}/auto-extract`; new `frontend/src/components/GlossaryPanel.jsx` (edit-in-place, auto/manual badges, LLM auto-extract via same OpenAI-compatible client as translator); `frontend/src/api/glossary.js` client; DubTab renders the panel when a job is loaded; `App.jsx` collects `glossaryTerms` and pipes them into `/dub/translate` on every call. Translator's existing `glossary` prompt injection (1.1) now actually gets data. |
| 1.4 | Dual subtitle export (original + translated together) | βœ… | Shipped 2026-04-21. `GET /dub/srt/{id}?dual=1` and `GET /dub/vtt/{id}?dual=1` stack translated line on top, original italicised underneath (`<i>…</i>`). Shared `_pick_subtitle_text()` helper so both formats stay identical. UI: "Dual subtitles" checkbox in Output Options (localStorage-persisted `omnivoice.dub.dualSubs`); SRT/VTT buttons show a ✦ badge + `_dual` filename suffix when on. |
**Exit criteria:** blind A/B on 3 clips (ENβ†’DE, ENβ†’JA, ENβ†’ES) shows β‰₯70 % preference for the new pipeline. Dub output materially feels more professional to an untrained listener.
---
## 🧱 Phase 2 β€” Foundation refactor _(10–14 weeks β†’ 🟑 partial)_
> *The invisible work that unblocks everything after Phase 4. Running in parallel where it doesn't interfere with Phase 1.*
Progress: **7/8 (88 %)** Β· Phase 2.5 has overshot expectations thanks to the design-system sprints; 2.8 was added after Phase 1 shipped to give the voice library its long-overdue first-class surface.
| ID | Item | Status | Notes |
|----|------|:---:|------|
| 2.1 | Persist the task queue (survive restart, SSE replay) | βœ… | Shipped 2026-04-21. New `jobs` + `job_events` tables (schema v4). New `backend/core/job_store.py` with CRUD + `sweep_orphans_on_startup()`. `TaskManager` now mirrors every state transition + event to disk; the in-memory queue stays as-is for speed. Startup lifespan flips any `pending`/`running` job to `failed` with a "restart interrupted" message. New endpoints: `GET /jobs`, `GET /jobs/{id}`, `GET /jobs/{id}/events`. `GET /tasks/stream/{id}?after_seq=N` now replays the persisted tail on reconnect before attaching to the live listener. Tests: `tests/test_job_store.py` (7 tests β€” lifecycle, event seq, per-job cap, active filter, startup sweep). Smoke suite extended with 4 jobs-endpoint tests. All 37 backend tests pass in 2.83 s. **Does not yet** persist the actual work β€” that's Phase 4.5 (step-level resumability); this phase only guarantees honest metadata across restarts. |
| 2.2 | Split the App.jsx monolith via Zustand store | βœ… | **5 slices shipped** 2026-04-20/21. `prefsSlice` (translateQuality/dualSubs/reviewMode), `glossarySlice`, `uiSlice` (mode + nav + sidebar tab + cheatsheet flag + voice-profile nav actions), `dubSlice` (18 pipeline fields with React-style setters), `generateSlice` (text/refText/instruct/language + 10 production-override knobs + vdStates). Persist middleware key bumped to v3 with the generate knobs + sidebar state. App.jsx `useState` count: **~60 β†’ 41** β€” everything remaining is File/Blob refs, backend listings (history/dubHistory/exportHistory/studioProjects/profiles), transient recording timers, and compare-modal state (all non-migratable by design). Children (DubTab/Sidebar) read from store directly; ~35 props dropped from the call sites. |
| 2.3 | TypeScript migration (incremental: store + API layer first) | βœ… | Shipped 2026-04-20 β€” **full API layer now TS**. All 8 remaining `.js` files (dub/engines/exports/generate/glossary/profiles/projects/system) migrated to `.ts` with typed response shapes sourced from a new shared `api/types.ts` (EngineBackend, Profile, ProfileUsage, GlossaryTerm, ProjectDetail, SystemInfo, ModelStatus, …). Call sites unchanged; every consumer now gets compile-time checks on API response shapes. Store slices + `api/client.ts` already TS from Phase 2.3's first pass. Component conversion to `.tsx` stays optional per-file. |
| 2.4 | Split `dub_core.py` into router + `services/dub_pipeline.py` | βœ… | Shipped in two passes 2026-04-21. `services/dub_pipeline.py` now holds: file-hash / path-safety / SSE helpers, `find_cached_job`, process lifecycle (`register_proc`/`unregister_proc`/`kill_job_procs`), in-memory + SQLite job state (`get_job`/`put_job`/`save_job`), subprocess wrapper factory (`run_proc_factory`), yt-dlp downloader, AND the full `ingest_pipeline` async generator (the downloadβ†’extractβ†’demucsβ†’sceneβ†’thumbnail prep flow). Module-level state (`_dub_jobs`, `_active_procs`, `_active_procs_lock`) moved too. `dub_core.py` keeps backward-compat aliases. **Size: 889 β†’ 525 lines (βˆ’40.9 %)**. 50 backend tests green (2.84 s). Transcription generators still in router β€” that's the remaining Phase 4 prerequisite, not a blocker. |
| 2.5 | Design-system primitives | βœ… | **MASSIVELY overshot.** 14 primitives: Button, Panel, Field/Input/Textarea/Select, Dialog, Slider, Badge, Tabs, Segmented, Tooltip, Progress, Menu, Table. 9 components migrated. Inline-style migration (separate design-track row) has since drained DubTab (93β†’2), Header (24β†’1), Sidebar (21β†’8), CloneDesignTab (33β†’0). Sessions ran 2026-04-19 β†’ 2026-04-21. |
| 2.6 | Logging + telemetry baseline | βœ… | Shipped 2026-04-21. `print()` sweep in `services/model_manager.py` β€” all four calls now use `logger.info` (model load, torch.compile apply/skip, idle unload). New `OMNIVOICE_JSON_LOGS=1` env flag swaps every handler to a single-line JSON formatter (verified: `{"t": "...", "level": "INFO", "name": "omnivoice.api", "msg": "..."}`), including the rotating file handler. Per-stage Prometheus counters at `/metrics` deferred β€” land when a real observability need arises. |
| 2.7 | Test floor ("every bug ships a regression test") | βœ… | Shipped 2026-04-21. New `tests/test_dub_pipeline_state.py` (8 tests β€” path safety, SSE shape, process tracking, in-memory + disk job round-trip) + `tests/test_translator.py` (5 tests β€” glossary preamble, no-LLM graceful fallback, empty-literal passthrough, full 3-step chain with mocked LLM, reflect-failure path). Total backend tests: **50** (up from 26) passing in 3.10 s. Integration tests covering the GPU / ASR model path stay out of CI β€” those are manual fixture-clip runs. |
| 2.8 | Voice profile page | βœ… | Shipped 2026-04-21. New `mode: 'voice'` + `activeVoiceId` in `App.jsx`; lazy-loaded `frontend/src/pages/VoiceProfile.{jsx,css}` (hero + details + try-it + usage). Backend: new `GET /profiles/{id}`, `PUT /profiles/{id}`, `GET /profiles/{id}/usage` (scans `studio_projects.state_json` for dub-segment counts). Sidebar gets an "Open" action on clone/design cards that routes here; Back returns to the previous mode. Design target: [`design/03-voice-library.md`](design/03-voice-library.md). |
**Exit criteria:** server can restart mid-dub without losing work. Frontend type-checks. No router file >300 lines. App.jsx ≀300 lines. CI blocks regressions.
---
## πŸ”Œ Phase 3 β€” Pluggable engines _(shipped 2026-04-21 β†’ βœ…)_
> *Stop being locked to one model family. Unlock VoxCPM2 as a serious alternative.*
Progress: **4/4 (100 %)**
| ID | Item | Status | Notes |
|----|------|:---:|------|
| 3.1 | TTS adapter interface | βœ… | Shipped 2026-04-21. `backend/services/tts_backend.py` β€” `TTSBackend` ABC + registry, `OmniVoiceBackend` wrapping the current model (zero behaviour change; reuses `model_manager.get_model()` so no double load), `list_backends()` with per-engine availability reasons, env-driven selection via `OMNIVOICE_TTS_BACKEND`. |
| 3.2 | Alternative TTS backends (VoxCPM2 + MOSS-TTS-Nano) | βœ… | Shipped 2026-04-21 / 2026-04-20. **VoxCPM2**: GPU studio pick, 30 langs, 48 kHz, `"(instruct)prompt"` syntax wired from OmniVoice's `instruct`. **MOSS-TTS-Nano-100M** added 2026-04-20 as the low-resource / broad-language pick β€” 100M-param autoregressive, 20 langs (incl. Arabic/Hebrew/Persian/Korean/Turkish), realtime on 4-core CPU, native 48 kHz stereo, Apache-2.0. Both `is_available()` return actionable install hints when deps are missing. **Runtime picker**: `POST /engines/select` + `backend/core/prefs.py` (atomic JSON store); Settings > Engines tab has **Use** buttons per family; env vars still override. 14 engine tests pass. |
| 3.3 | ASR adapter interface | βœ… | Shipped 2026-04-21. `backend/services/asr_backend.py` β€” `ASRBackend` ABC, `MLXWhisperBackend` (Apple Silicon, current default), `PyTorchWhisperBackend` (CUDA / CPU fallback using the TTS model's `_asr_pipe`). `active_backend_id()` auto-detects based on `torch.backends.mps` availability; override with `OMNIVOICE_ASR_BACKEND`. |
| 3.4 | LLM adapter | βœ… | Shipped 2026-04-21. `backend/services/llm_backend.py` β€” `LLMBackend` ABC, `OpenAICompatBackend` (lifts the Ollama/OpenAI client out of `translator.py` so glossary auto-extract + future Directorial AI share one code path), `OffBackend` (explicit no-LLM with a copy-paste env-var hint). Privacy default honoured: Cloud LLMs opt-in per-feature, never required. |
**Exit criteria:** engine swap works without restart on Mac + Linux. VoxCPM2 available when CUDA 12+ detected.
> **Exit status:** all four adapter interfaces shipped 2026-04-21 with the env-driven selection + registry + `/engines` HTTP surface. Backward-compat preserved β€” no existing callers migrated yet. Actual swap-without-restart (live reconfiguration) + the Settings-UI engine picker are follow-up UX tasks, tracked in Phase 2.2's ongoing App.jsx / Settings.jsx work.
---
## πŸŽ›οΈ Phase 4 β€” The two bets land _(shipped 2026-04-21 β†’ βœ…)_
> *The defining phase. This is why someone chooses OmniVoice over everything else. Built on Phase 2's persistent job store and Phase 3's adapters.*
Progress: **6/6 βœ…** (both bets wired end-to-end; review banners + step-level resumability live)
| ID | Item | Status | Notes |
|----|------|:---:|------|
| 4.1 | 🎲 **Incremental re-dub (Bet B)** | βœ… | Shipped 2026-04-21 end-to-end. **Service** (`services/incremental.py`) fingerprints each segment's generation inputs. **Backend** (`dub_generate.py`) now accepts `regen_only: [seg_ids]` + parallel `segment_ids: [...]`. When `regen_only` is set, listed segments run through TTS; unlisted segments reuse their on-disk `seg_N.wav` (resampled/padded as needed), then the mix reassembles over the full timeline. **UI**: App.jsx snapshots fingerprints after every successful dub via `/tools/incremental`; DubTab footer shows "N segments changed since last generate" warn badge when stale > 0; a **"Regen N changed"** pink-tone button appears alongside "Generate Dub" in the action footer when stale > 0, calling `handleDubGenerate({ regenOnly: incrementalPlan.stale })`. On partial-regen completion, fingerprints re-snapshot so the warn badge reflects the new clean state. |
| 4.2 | 🎲 **Directorial AI (Bet A)** | βœ… | Shipped 2026-04-21 end-to-end. **Backend:** `backend/services/director.py` β€” stable taxonomy (5 dimensions, 30+ values), dual parser (LLM via `OpenAICompatBackend` when configured, keyword-heuristic fallback), `Direction.instruct_prompt()`/`translate_hint()`/`rate_bias()`. Translator injects direction into both reflect + adapt prompts per segment via new `directions={seg_id: text}` param. `dub_generate.py` appends the parsed `instruct_prompt()` onto the per-segment `instruct_str` and multiplies `seg_speed` by `rate_bias()`. **Schema:** `DubSegment.direction` + `TranslateSegment.direction` added. **Frontend:** `DubSegmentRow` "…" menu gains a "Set direction…" entry with a ✨ icon; segments with direction show the ✨ badge in place of the "…". `components/DirectionDialog.jsx` is a Dialog with live `/tools/direction` preview showing taxonomy tokens + TTS instruct + translate hint + rate bias. `App.jsx` wires `openDirection`/`saveDirection` into Undo, passes `direction` through `handleTranslateAll` and `handleDubGenerate`. 5 unit tests pass. |
| 4.3 | Staged checkpoints (review banners) | βœ… | Shipped 2026-04-20. The dub flow has three natural review points β€” **post-ASR** (transcripts ready), **post-translate** (translations ready), **post-generate** (dub complete). `components/CheckpointBanner.jsx` renders phase-appropriate banner above the segment table with an inline **Continue β†’** button that triggers the next stage (Translate / Generate) directly. Stage auto-detected: `dubStep === 'editing'` + no `text_original` β†’ ASR; `editing` + any `text_original` β†’ translate; `done` β†’ final. Dismissible per-stage (won't reappear until reload). **Review mode toggle** added to `prefsSlice` (persisted via zustand/persist); Settings > Engines tab exposes a **Review between stages / Rapid-fire** segmented control. Defaults to 'on'. |
| 4.4 | Speech-rate engineering | βœ… | Shipped 2026-04-21 end-to-end. `backend/services/speech_rate.py` β€” CPS tables per language, tolerance window (0.92–1.08), LLM trim/expand loop with MAX_ATTEMPTS=3. **Wired into Cinematic translate:** `TranslateSegment.slot_seconds` auto-populated by `App.jsx` from `seg.end - seg.start`. When cinematic + a slot are supplied, translator runs `adjust_for_slot` on each refined line; returns `rate_ratio` + optional `rate_error` per segment. **UI**: `DubSegmentRow` now renders a compact πŸ“– `1.12Γ—` badge in the time column whenever `seg.rate_ratio` drifted >3% from 1.0; colour-coded (red above 1.15, blue-info below 0.85, muted otherwise) with a tooltip showing the error if rate-fit bailed. `POST /tools/rate-fit` + `ToolsPage.RateFitTool` remain for standalone use. |
| 4.5 | Step-level resumability | βœ… | Shipped 2026-04-20 end-to-end. Per-segment fingerprints are persisted in `job.seg_hashes` **after each successful segment gen** (flushed every 8 segs via `_save_job`) so a kill -9 mid-dub loses at most the in-flight segment. The `done` SSE event now streams `seg_hashes` to the client, which snapshots them as `lastGenFingerprints` without a follow-up `/tools/incremental` round-trip. `loadProject` rehydrates from `project.segHashes`; `restoreDubHistory` rehydrates from `job.seg_hashes`. Net effect: closing the app mid-dub + reopening lights up the "Regen N changed" button for the exact residual, and the user resumes from the last good segment instead of restarting. |
| 4.6 | Headless CLI + standalone Tools page | βœ… | Shipped 2026-04-21. New `omnivoice-dub` CLI (`omnivoice/cli/dub.py`, `pyproject.toml` entry point) β€” drives the full pipeline over HTTP, stream-reads SSE tasks, supports `--url` ingest, `--quality cinematic`, `--voice`, `--glossary`. New `backend/api/routers/tools.py` with `/tools/probe`, `/tools/incremental`, `/tools/direction`, `/tools/rate-fit`. New `frontend/src/pages/ToolsPage.jsx` wired at `mode='tools'`, hosts 3 interactive cards (Directorial AI demo, Rate-fit demo, ffprobe). 3 smoke tests added. |
**Exit criteria:** changing one word regenerates in <5 s on the fixture clip. Directorial evals beat baseline by β‰₯15 % on the rubric. Kill -9 mid-dub resumes to completion.
> **Exit status:** all six sub-phases shipped with integration into the main dub flow + DubTab workspace (2026-04-20/21). The **measurable** exit criteria (5 s incremental wall-clock, +15 % directorial eval, benchmarked crash recovery) still require a fixture-backed eval sprint β€” the code path is live, the numbers are not yet pinned. Tracked separately under the Quality track.
---
## πŸ—οΈ Phase 5 β€” Productisation _(demand-driven β†’ 🚫 deferred)_
> *Do not build proactively. Trigger-based only.*
Progress: **0/5 (0 %)**
| ID | Item | Trigger that starts it | Status |
|----|------|------|:---:|
| 5.1 | Multi-worker + Redis/Postgres job queue | First team asks to share a GPU across users | 🚫 |
| 5.2 | OpenTelemetry tracing | First "why is this slow?" incident we can't diagnose | 🚫 |
| 5.3 | Optional auth (magic-link / passkey) | First team-deployment request | 🚫 |
| 5.4 | Signed installers + auto-update (Tauri) | Paid release or pre-order list forms | 🚫 |
| 5.5 | Plugin SDK (third-party TTS/ASR/LLM) | First external contributor ships an adapter | 🚫 |
None on the critical path to world-class. All are answers to real demand.
---
## πŸ›€οΈ Parallel tracks (always running)
### 🎨 Design track _(ongoing β†’ 🟑 very active)_
| Item | Status | Notes |
|------|:---:|------|
| Motion language (60 fps, spring easing, staggers) | 🟑 | Tokens shipped in `ui/tokens.css` (durations, easings); launch animation live; stagger + spring on interactions still pending. |
| Density dial (Small/Normal/Max rebuilds layout, not just font-size) | ⏳ | Current S/M/L only scales font size. |
| Error messages rewritten in every path | 🟑 | **20 done** (Phase 0 top-10 + round-2 on 2026-04-21): `system.py` (log-read + log-clear), `dub_core.py` (invalid job_id Γ—2, invalid url, yt-dlp missing, ASR not loaded), `profiles.py` (nothing-to-update + profile-not-found Γ—2), `dub_export.py` (no-such-job, ffmpeg mux fail Γ—2, MP3 encode fail), `glossary.py` (empty PUT body). All now include what-happened / why / what-to-do. 138 backend tests green. ~15 lower-traffic paths remain. |
| Onboarding by doing (pre-loaded sample project) | ⏳ | See 0.6 β€” deferred. |
| Launch animation + recognisable sound | 🟑 | Animation live; sound not yet. |
| Weekly inconsistency audit (1 h / Friday) | ⏳ | Cadence not yet established. |
| Design-system primitives (14) | βœ… | Full inventory above. |
| Migrate remaining inline styles | 🟑 | **Four biggest offenders drained 2026-04-20** β€” DubTab **93 β†’ 2**, Header **24 β†’ 1**, Sidebar **21 β†’ 8**, CloneDesignTab **33 β†’ 0**. All remaining are genuinely dynamic (per-row `--row-accent` CSS custom props in Sidebar, per-bar `height/animationDelay` in WaveBars, `opacity` computed from index in skeleton rows, `fontSize` by prop). New class systems: `.dub-*` (DubTab), `.hq-col-*/.hq-stats__*/.hq-logo-*` (Header), `.sidebar-tile--*/.sidebar__scroll/.history-*--*` (Sidebar), `.clone-*/.label-row--*` (CloneDesignTab). Drag-hover on `.file-drag` and `.dub-idle-drop` now toggles `.is-dragging` instead of mutating styles via DOM. Remaining 119 across the tail (Launchpad, KeyboardCheatsheet, DubSegmentRow, WaveformTimeline, etc.) β€” less concentrated, lower-leverage. |
### ⚑ Performance track _(⏳ not started)_
| Item | Status | Current measurement |
|------|:---:|------|
| Batched TTS (8–16 segments per forward pass) | ⏳ | 1 segment per call today. |
| Kill per-segment disk round-trip | ⏳ | `dub_generate.py:132-133` saves + re-reads per segment. |
| Cold start ≀1.5 s to first audible sample | ⏳ | Currently 4+ s on Apple Silicon. |
| Speculative regeneration on hover | ⏳ | β€” |
| Crash-sandbox engines (subprocess isolation) | ⏳ | Single CUDA OOM still kills server. |
| Interaction budgets (<50 ms UI, <200 ms preview, <4 s first seg) | ⏳ | Not measured. |
| Dedicated dev-week per quarter | ⏳ | Cadence not yet booked. |
### ✨ Feature-magic track _(⏳ not started)_
| Feature | Status | Phase gate |
|------|:---:|------|
| Project-level casting view (drag voices to speakers) | ⏳ | After Phase 3 |
| Voice memory across projects | ⏳ | After Phase 4 |
| Context-aware pipeline (video frames β†’ pipeline decisions) | ⏳ | After Phase 4 |
| On-device learning from corrections (user edits β†’ LoRA) | ⏳ | Research only; possibly Phase 5+ |
| Real-time dub preview (stream TTS as you edit) | ⏳ | After Phase 4.1 |
### πŸ§ͺ Quality track _(🟑 underway)_
| Item | Status | Notes |
|------|:---:|------|
| Every bug ships a regression test | ⏳ | Rule written, not yet enforced in CI. |
| Perf regression budget (≀5 % on fixture clip) | ⏳ | No fixture clip yet. |
| Accessibility (keyboard-first, WCAG AA, ARIA live regions) | 🟑 | Focus rings token defined; full audit pending. |
| Privacy (zero telemetry by default, per-feature opt-in) | βœ… | Enforced in Settings β†’ Privacy tab. |
| Docs updated per phase | 🟑 | STRUCTURE.md, ROADMAP.md, ui/README.md, research/LEARNINGS.md, design/*.md all current. |
---
## ⚠️ Risks & unknowns
| Risk | Status | Mitigation in place |
|------|:---:|------|
| Local LLM β‰  frontier LLM for translation | ⏳ open | Plan: accept "Cinematic w/ local" + "Cinematic w/ cloud (opt-in)" as two SKUs. Never require cloud. |
| Model landscape drift (VoxCPM3, F5-TTS-v3 during roadmap) | ⏳ open | Phase 3 adapter interface is the hedge. |
| Tauri code-signing / notarisation costs | ⏳ open | Budgeted into Phase 5.4. 2-week compliance headroom. |
| pyannote HF-token friction (bounces new users on install) | ⏳ open | Plan: "skip diarisation for now" path; auto-diarise on first token-present run. |
| On-device fine-tuning quality drift | ⏳ open | Gated: LoRA only ships with automatic eval gate. |
| Voice-cloning likeness / IP concerns | ⏳ open | Log every clone with source + consent prompts. Deletion tools in Phase 5. |
| Apple MPS parity with CUDA | 🟑 mitigating | Adapter interface + per-engine capability gates. VoxCPM2 gated to CUDA only in UI. |
| Solo-dev bandwidth | 🟑 mitigating | 20 % buffer on every estimate; phases can slip without changing order. |
---
## πŸ“ Success metrics
| Metric | Target | Current reading | Status |
|------|------|------|:---:|
| Dub quality (blind pref vs. VideoLingo + pyVideoTrans, 10 clips) | β‰₯70 % | not measured | ⏳ |
| Time to first dub (fresh install β†’ 3-min YouTube clip) | ≀10 min | not measured | ⏳ |
| Cold start (model load β†’ first audible sample) | ≀1.5 s | ~4 s | 🟑 |
| Incremental re-dub (single-word change β†’ audio on timeline) | ≀5 s | **1.96 s** on sentence-level segs (Apple Silicon MPS, num_step=8 preview path, 30-seg fireship clip) Β· 70Γ— speedup vs full Β· TTS = 96 % of budget | βœ… |
| Recovery (`kill -9` mid-dub β†’ resume to completion) | zero data loss | seg_hashes persistence live (4.5); benchmark unmeasured | 🟑 |
| Directorial eval (natural-language direction vs. baseline rubric) | +15 % | code path live (4.2); eval fixtures unmeasured | 🟑 |
| Engine robustness (engine crash β†’ auto-recovery) | jobs continue | ❌ full-server crash | ⏳ |
| Interaction latency p95 | <50 ms UI, <200 ms preview | not measured | ⏳ |
| Regression safety (shipped-bug β†’ regression test in last quarter) | 100 % re-surface rate 0 | n/a (no incidents yet) | βœ… |
| External contributor PR in first week | 3 successive external PRs | 0 (private repo) | ⏳ |
---
## πŸ“ Changelog
Reverse-chronological shipping log. Every phase exit + material sub-phase adds a row.
### 2026-04-21 β€” Auto-speaker-clone + pro-grade UI consolidation
The pro-grade dubbing promise: "same speaker in a new language." Before this change the user had to manually record or upload a reference voice and assign it to every segment. Now the reference is extracted automatically from the source video's own vocals track.
**Backend:**
- New `services/speaker_clone.py` (~150 LOC). For each diarised speaker, picks the longest-first subset of segments that accumulates into 5–15 s of clean audio from `vocals.wav` (Demucs output, not the raw mix β€” isolated from BG music). Writes `dub_jobs/{id}/voice_speaker_N.wav` + concatenated transcript text. Skips speakers with <5 s total β€” better to fall back than ship a thin clone.
- `dub_core` transcribe pipeline calls it after diarisation, stores results at `job["speaker_clones"]`, assigns `segments[i].profile_id = "auto:speaker_N"` by default (user-assigned profiles win).
- `dub_generate._gen` resolves the `auto:` prefix: reads the job-scoped ref audio + ref text instead of hitting the persistent `voice_profiles` table.
- SSE `final` event now streams `speaker_clones` to the frontend.
- Smoke-tested: synthetic 20 s vocals β†’ correctly picks the long segments, skips speakers with <5 s, writes a valid WAV.
**Frontend (pro-grade UI consolidation):**
- `dubSlice.speakerClones: Record<speakerId, {ref_audio, ref_text, duration, source_count}>`. Transcribe `final` handler populates it.
- **CAST strip** (left column, under video): renamed from "Speaker Voices" to "CAST". Each speaker's dropdown gains a first-class `🎀 From video Β· 7.8s` option when an auto-clone exists β€” pre-selected on segments by default.
- **Removed "Apply Voice to All"** row (right column). It was a degenerate case of CAST when there's only one speaker; now the multi-speaker and single-speaker paths share one control.
- **Waveform timeline**: `height: 64` hard-cap β†’ `height: 'auto'`. Reclaims ~80 px of dead space below the 3-tile strip. The waveform now fills whatever vertical room the left panel has.
- Typecheck clean, 62 backend tests pass, build 236 ms.
**What's still rough** (honest):
- Waveform still renders as fixed tile segments rather than one continuous timeline. Deeper fix is a proper regions overlay on one WaveSurfer instance β€” v2.
- Segment-table column dedupe (when every row's SPKR / LANG / VOICE are identical, hide those columns) is still pending β€” requires DubSegmentTable restructure.
- No "Save clone as persistent profile" button yet β€” job-scoped only.
### 2026-04-21 β€” Phase 4.1 truly βœ… on sentence-level segs: **1.96 s / 70Γ— speedup**
Re-benched after the segmentation fix, with num_step dropped from 16 β†’ 8 on the partial-regen path only (full-dub + export still run at 16). The change is one line in `dub_generate.py`:
```python
_num_step = 8 if regen_only is not None else req.num_step
```
**Verdict:** server total = **1.95 s** Β· TTS = 1.89 s Β· cache + mix + save = 0.05 s combined Β· client wall-clock = 1.96 s. Target ≀5.0 s βœ…. Speedup vs full (at num_step=16) = 70Γ—.
**The 22 s client/server gap from the previous run disappeared too.** It was one-time "first task after full-dub" warm-up noise, not a persistent tax. Not investigating further β€” the number is now what the roadmap promised.
**Honest tradeoff (now handled end-to-end):** num_step=8 costs ~10–20 % perceptual quality vs num_step=16. Shipped 2026-04-21:
- **Backend:** `DubRequest` gained an explicit `preview: bool` flag. num_step is derived: `_num_step = 8 if req.preview else req.num_step`. Decoupled from `regen_only`. Per-seg num_step is persisted in `job.seg_num_step` (mirror of `seg_hashes`) and streamed to the client in the `done` event.
- **Frontend:** `dubSlice` tracks `previewSegIds` (the ids whose num_step < the current `steps` floor). The "Regen N changed" button passes `preview: true`; the `done` handler populates `previewSegIds` from `evt.seg_num_step`. Export handlers (MP4 / WAV / MP3 / Clips / Stems) now pre-flight via `finalizeTtsBeforeExport()` β€” if any preview segs are outstanding, they're re-rendered at full quality (`preview: false`) before the download URL is triggered. Subtitle exports (SRT / VTT) skip the pre-flight since they don't depend on audio quality.
- User flow: edit word β†’ Regen (1.96 s, preview quality) β†’ optional further edits β†’ Export MP4 (toast "Upgrading 1 preview-quality segment to full quality…", ~15 s TTS, then browser download). Feels fast during editing, exports at full quality.
- 43 backend tests green, typecheck clean, build 198 ms.
### 2026-04-21 β€” Segmentation regression caught: 489 segs β†’ ~36 (13Γ— fewer)
User noticed the segment table was rendering **single-word rows** ("ersten", "Runde", "eine"…) instead of sentences. Old code produced 8 segs for a 36 s clip; current produced 100+. The bench previously surfaced "489 segments for a 3-min clip" but I treated it as "that's just what Whisper returns" β€” it wasn't.
Adding instrumented log lines to the transcribe pipeline pinned the stage:
```
seg-trace chunk=1/5 raw=9 after_segment_transcript=7 after_subtitles=60
seg-trace chunk=2/5 raw=8 after_segment_transcript=7 after_subtitles=117
seg-trace chunk=3/5 raw=7 after_segment_transcript=7 after_subtitles=120
seg-trace chunk=4/5 raw=8 after_segment_transcript=8 after_subtitles=101
seg-trace chunk=5/5 raw=13 after_segment_transcript=7 after_subtitles=91
```
**Cause:** `services/subtitle_segmenter.segment_for_subtitles` enforced Netflix's **17 CPS reading-speed ceiling** as a segmentation rule. Normal German speech runs 15–25 CPS, so the splitter fired on every sentence and recursed until single words. It's a subtitle *display* rule being applied as dubbing *segmentation* β€” different units entirely.
**Fix (surgical):** removed both call sites of `segment_for_subtitles` from the transcribe pipeline (`dub_core.py` line 377 + 513). `segment_transcript` already produces sentence-level output with its own duration/char caps (IDEAL_DUR=4.5s, MAX_DUR=9s, MAX_CHARS=140). Dropped the now-unused import. The subtitle CPS rule stays available in `subtitle_segmenter.py` for a future SRT-export path where it's appropriate.
**Expected result:** 3-min clip β†’ ~36 segments (matches user's "8 sentences in 36s" memory β†’ ~4.5 s/seg). 63 tests pass.
### 2026-04-21 β€” Phase 4.1 benchmark PASSES at 4.04 s warm + 2 UI regressions fixed
Second bench run with server-side profile instrumentation. Same fixture, same flow, one small change: the model was warm from the prior full-dub pass (the real-user flow β€” you dub first, then edit).
| Stage | Cold (first run) | Warm (real UX) |
|---|---|---|
| Incremental regen | 7.05 s | **4.04 s** βœ… |
| speedup vs full | 10.5Γ— | 18.5Γ— |
**Server-side profile (`bench[incremental]` log line):**
```
total=4.02s cache=0.01s tts=3.97s mix=0.00s save=0.01s segs=30 regen=1
```
TTS is 99 % of the budget. Cache load (29 WAVs), timeline mix (30 segments with fades + gain), and file save (24 kHz mono WAV across 3 min of timeline) combined run in <20 ms. The Phase 4.1 incremental machinery is fast β€” the wall-clock floor is whatever it takes to synthesise one new segment.
**Conclusion:** The ≀5 s target matches real-world UX (you dub once, then edit β€” model warm). Cold-path 7 s is a first-ever load, not a realistic regen. Phase 4.1 exit criterion βœ… verified.
**Two regressions caught when the user opened the UI after the session's refactors:**
1. `zustand persist` threw "couldn't be migrated" because I bumped `version: 1 β†’ 2 β†’ 3` without a `migrate` fn. Fixed: passthrough migrate accepts any version ≀3, relying on slice defaults for new fields.
2. `<DubTab>` referenced `onGlossaryChange` which I'd stripped from the prop destructure during the store-direct refactor. Fixed: restored to the destructure block.
### 2026-04-21 β€” Phase 4.1 benchmarked on fireship clip: **7.05 s, MISS by 2 s**
First real measurement of the Phase 4.1 exit criterion. Fixture: `https://www.youtube.com/watch?v=ZzI9JE0i6Lc` (fireship clip, ~3 min, 489 transcribed segments β€” capped to first 30 for tractable baseline). Apple Silicon MPS, OmniVoice default engine.
| Stage | Wall-clock |
|---|---|
| Ingest URL β†’ prep (cold first run) | 80.7 s |
| Ingest URL β†’ prep (cache hit, same URL) | 4.0 s |
| Transcribe 3-min clip β†’ 489 segments | 74–137 s |
| Full dub (30 segs, cold model) | **74.33 s** |
| Incremental regen (1 of 30 edited) | **7.05 s** |
| Speedup incremental vs full | 10.5Γ— |
**Target ≀5.0 s Β· Actual 7.05 s Β· MISS by 2.05 s.**
The incremental logic works β€” 10.5Γ— speedup confirms cached seg_N.wav files are being reused. The 2 s gap to target is likely disk I/O (29 cached WAVs loaded sequentially) + timeline mix + `sync_scores` computation on cached segs that never change. Lowest-hanging fixes: parallel cached-WAV load, skip no-op resample when `cached_sr == model.sampling_rate`, skip sync_scores on cached segs.
**Honest ROADMAP update:** Phase 4.1 exit criterion ⚠️ `🟑 β†’ ❌`. Code path works; target wasn't pinned. Two paths forward:
1. Accept 7 s as the real number, widen target to "≀10 s on typical M-series".
2. Optimize the three hotspots above and retry the bench.
Either way: the `βœ…` on the Phase 4.1 row was premature β€” it meant "code shipped", not "benchmark passed."
### 2026-04-21 β€” Phase 4.1 benchmark harness + latent ingest bugs caught
Running Karpathy's "define a verifiable goal" principle on Phase 4.1's `≀5 s incremental` exit criterion surfaced **four latent `NameError` bugs in the ingest pipeline** that had never been hit in tests (the tests never drive the full yt-dlp β†’ extract β†’ demucs β†’ scene path). All in `backend/services/dub_pipeline.py` after Phase 2.4's refactor β€” the symbols were referenced but never imported:
- `asyncio` (used by 14 call sites in the file)
- `find_ffmpeg` from `services.ffmpeg_utils`
- `_get_semaphore` + `_spawn_with_retry` from `services.ffmpeg_utils`
- `shutil`, `re`, `sys`, `soundfile as sf`, `AsyncIterator`, `get_best_device`, `HTTPException` (collateral of the move)
Also caught by the same `pyflakes` pass: `backend/services/translator.py` uses `Optional` in two signatures without importing it. Fixed.
**New file:** `scripts/bench_incremental.py` β€” 140-line HTTP-driven harness that drives the full pipeline on a fixture YouTube URL, times full-dub and incremental regen-only, and reports pass/fail vs. the 5 s target. Drives `POST /dub/ingest-url` β†’ `/tasks/stream/{task_id}` SSE β†’ `/dub/transcribe-stream/{job_id}` β†’ `POST /dub/generate/{job_id}` (twice: full then `regen_only`).
Lesson: the "last mile" integration of Phase 4.1 was never end-to-end tested. Marking it βœ… on the roadmap was premature β€” the service + HTTP surface worked in isolation, but the real user path (URL β†’ dub β†’ edit β†’ regen) had never actually been run.
### 2026-04-21 β€” Phase 2 closes: generateSlice + uiSlice expansion
- **`generateSlice.ts`** β€” new slice owns all 14 generate-tab fields: `text`, `refText`, `instruct`, `language`, plus the 10 production-override knobs (`speed`, `steps`, `cfg`, `tShift`, `posTemp`, `classTemp`, `layerPenalty`, `denoise`, `postprocess`, `duration`) and the `vdStates` category map. Users' synthesis prefs now persist across reloads via the store's `partialize`.
- **`uiSlice`** grew β€” added `sidebarTab`, `isSidebarProjectsCollapsed`, `showCheatsheet` (with the functional-updater pattern). Persisted.
- App.jsx `useState` count: ~60 β†’ **41**. Remaining are all non-migratable by design β€” File/Blob refs, backend-loaded listings, transient recording timers, compare-modal state.
- Store composition now: 5 slices (prefs, glossary, ui, dub, generate) under the same `persist` middleware. Persistence key bumped to v3. Typecheck clean, build 225 ms.
- **Phase 2.2 marked βœ…** β€” the monolith split is functionally complete. Phase 2 is 8/8.
### 2026-04-20 β€” Header + Sidebar + CloneDesignTab inline styles drained
Four biggest inline-style offenders in the repo, all migrated in one pass:
- **Header 24 β†’ 1.** `.header-area` now a grid via CSS (not inline). New class system: `.hq-col-left/center/right`, `.hq-logo-*`, `.hq-stats__key/--gpu-active/__sep/__status-*`, `.hq-breadcrumb-sep`, `.hq-view-icon`, `.hq-flush-btn`, `.hq-reload-btn`. `.hq-wave.is-active` replaces the inline opacity flip. Only the per-bar dynamic style (WaveBars `height`/`animationDelay`) stays inline.
- **Sidebar 21 β†’ 8.** Collapsed-mode sidebar tiles get a `.sidebar-tile` primitive with color variants (`--audio/--clone/--design/--success`). Body scroll container β†’ `.sidebar__scroll.is-collapsed`. History-item variants get proper classes (`.history-item--dub`, `.history-kind--audio`, `.history-meta--locked`, `.history-title--clamp`, `.history-subtitle--seed/--italic`, `.history-audio`). Remaining 8 are all dynamic `--row-accent` CSS custom props (per-item colors).
- **CloneDesignTab 33 β†’ 0 βœ….** Full grid layout, label-row variants (`--center/--spread/--flush/--sm`), drop-zone drag-hover now toggles `.is-dragging` (class added to `.file-drag` in index.css with `:hover` equivalent). Sliders column, Production Overrides column, and the duration input all classed.
- Typecheck clean, build 209 ms. Repo-wide inline styles: ~186 β†’ ~119 (remaining tail is low-leverage, spread across 17 components).
### 2026-04-20 β€” DubTab inline styles drained (93 β†’ 2)
- **DubTab went from 93 inline-style objects to 2** (both unavoidable dynamics: per-row opacity in the idle skeleton, and `fontSize: large ? … : …` in `PrepOverlay`). Repo-wide inline-styles: 186 β†’ 97 (-48%).
- New utility classes: `.dub-col`, `.dub-panel-col`, `.dub-split-1/-2`, `.dub-idle-drop` (+ `.is-dragging` for drag hover), `.dub-ingest-row`, `.dub-cast`, `.dub-gen-overlay`, `.dub-skel-*` family for the placeholder segment rows, `.dub-settings-bar`, `.dub-transcript-*`, `.dub-bulk-select`, `.dub-outputs-*`, `.dub-footer-*`.
- Drag-hover no longer mutates `e.currentTarget.style` directly β€” it toggles `.is-dragging`. Cleaner React idiom + lets designers retarget the hover look without touching the handler.
- Typecheck clean, build 205 ms.
### 2026-04-20 β€” Store-direct reads in DubTab + Sidebar
- **DubTab now reads ~30 fields from the store directly** β€” dropped from the App.jsx prop interface: `dubJobId`, `dubStep`, `setDubStep`, `dubPrepStage`, `dubFilename`, `dubDuration`, `dubSegments`, `setDubSegments`, `dubTranscript`, `dubLang`, `setDubLang`, `dubLangCode`, `setDubLangCode`, `dubInstruct`, `setDubInstruct`, `dubTracks`, `dubError`, `dubProgress`, `isTranslating`, `preserveBg`, `setPreserveBg`, `defaultTrack`, `setDefaultTrack`, `exportTracks`, `setExportTracks`, `activeProjectName`, `isSidebarCollapsed`, `setIsSidebarCollapsed`, `translateQuality`, `setTranslateQuality`, `dualSubs`, `setDualSubs`. App.jsx's `<DubTab>` JSX went from 47 prop attributes to 22.
- **Sidebar** dropped 4 store-owned props (`mode`, `isSidebarCollapsed`, `dubStep`, `activeProjectId`) and reads them directly. Saves prop drilling from App β†’ Sidebar across every mode switch.
- Store is now the source of truth β€” App.jsx is the orchestrator (handlers + side-effectful state), children read from the store. Handlers stay prop-threaded because they close over App.jsx's scope (API clients, refs, timer state).
- Typecheck clean, build 157 ms.
### 2026-04-20 β€” Phase 2.2 dubSlice migration (the big one)
- **`dubSlice.ts`** now owns all 18 dub-pipeline state fields. React-style setters (`(v | prev => next)`) via a tiny `resolve()` helper β€” every existing functional-update call site keeps working. Not persisted: project-load + dub-history restore explicitly rehydrate.
- App.jsx `useState` count for dub state: **18 β†’ 0**. Only non-serialisable things stay local (File, Blob URLs, timer refs, `dubHistory` listing, `showTranscript` toggle, `previewAudios` cache).
- Setup for future work: deep children (DubTab, DubSegmentTable) could now read state directly from the store instead of receiving 30+ props β€” that's a separate refactor pass. For this commit, App.jsx still threads props so no child contract changed.
- Store composition: 4 slices + persist middleware. Typecheck clean, build 152 ms.
### 2026-04-20 β€” Phase 2.2 uiSlice migration
- **New `uiSlice.ts`** covers the "where am I in the app?" state cluster: `mode`, `activeProjectId`, `activeProjectName`, `activeVoiceId`, `modeBeforeVoice`, `isSidebarCollapsed`, `uiScale`. Action methods `openVoiceProfile(id)` / `closeVoiceProfile()` replace the tangled setMode/setModeBeforeVoice/setActiveVoiceId coordination that used to live in App.jsx.
- App.jsx dropped **8 `useState` calls** for these fields + the two useCallback wrappers around voice-profile nav. Legacy `omni_ui` localStorage restore still writes via the store's setters, so existing users' uiScale/mode/sidebar state migrates on first load.
- `setActiveProjectId` + `setActiveProjectName` unified into one `setActiveProject(id, name?)` action β€” four call sites simplified.
- Store persistence key bumped to v2; new partialize fields: `mode`, `isSidebarCollapsed`, `uiScale` (active project/voice ids stay transient β€” reload returns you to launchpad rather than half-loading stale state).
- Typecheck clean, build 160 ms.
### 2026-04-20 β€” Phase 2.3 API layer fully on TypeScript
- **All 8 remaining API `.js` files migrated to `.ts`.** New shared `api/types.ts` exports typed shapes: `EngineBackend` / `EngineFamilyResponse` / `SelectEngineResponse`, `Profile` / `ProfileUsage`, `ProjectSummary` / `ProjectDetail`, `GlossaryTerm` / `AutoExtractResponse`, `SystemInfo` / `ModelStatus` / `LogsResponse`, `DubHistoryResponse` / `DubTranslateResponse`.
- Every API wrapper now has a typed return signature β€” consumers across App.jsx, Settings, Sidebar, DubTab, VoiceProfile, etc. get compile-time checks on response shapes without changing call sites (thanks to `allowJs: true`).
- Typecheck clean, build 209 ms. Store slices + `api/client.ts` already TS from Phase 2.3's first pass; the only remaining Phase 2.3 follow-up is converting component `.jsx` files to `.tsx` (optional, can happen file-by-file).
### 2026-04-20 β€” Phase 4.3 staged checkpoint banners
- **Phase 4 complete.** 6/6 sub-phases shipped. The three natural review points in the dub flow (post-ASR / post-translate / post-generate) now have explicit UI gates.
- **`CheckpointBanner` component** renders above the segment table with a stage-appropriate headline + an inline **Continue β†’** button (Translate after ASR, Generate after translate). Stage auto-detected from `dubStep` + whether any segment has `text_original`. Dismissible per-stage.
- **`reviewMode` pref** added to `prefsSlice` (persisted). Defaults to 'on'. Settings > Engines exposes a **Review between stages / Rapid-fire** segmented control β€” power-users flip to Rapid-fire to go straight from Prepare β†’ Translate β†’ Generate without nudges.
- Typecheck clean, build 206 ms.
### 2026-04-20 β€” Phase 4.5 + MOSS-TTS-Nano engine + runtime picker
- **4.5 Step-level resumability βœ….** `job.seg_hashes` now persists per-segment fingerprints after each successful gen (flushed every 8 via `_save_job`). The `done` SSE event streams them to the client, which snapshots `lastGenFingerprints` directly β€” no `/tools/incremental` round-trip. `loadProject` + `restoreDubHistory` rehydrate from `project.segHashes` / `job.seg_hashes`. Kill -9 mid-dub β†’ reopen β†’ "Regen N changed" lights up for the exact residual.
- **MOSS-TTS-Nano-100M (3rd TTS adapter).** New `MossTTSNanoBackend` in `services/tts_backend.py` β€” 100M-param autoregressive model, 20 languages (incl. Arabic/Hebrew/Persian/Korean/Turkish), realtime on 4-core CPU, native 48 kHz, Apache-2.0. Fills the "runs on a fanless laptop" + "commercial-safe broad-language" niche next to OmniVoice (600+ langs) and VoxCPM2 (30 langs). Lazy-loaded, `is_available()` points users to the install path.
- **Runtime engine picker.** `backend/core/prefs.py` (atomic JSON store under DATA_DIR). `active_backend_id()` in all three families (TTS/ASR/LLM) now reads env var β†’ prefs.json β†’ default; env still wins. New `POST /engines/select` refuses unknown families + unavailable backends. Settings > Engines tab renders a **Use** button per engine β€” click to switch, choice persists across restarts. New `selectEngine(family, backend_id)` helper in `frontend/src/api/engines.js`.
- 50/51 tests pass (1 pre-existing skip); typecheck clean; frontend build 419 ms.
### 2026-04-21 β€” Phase 4.1 partial-regen + 4.4 polish
Bet B now runs end-to-end.
- **4.1 Incremental re-dub β€” partial regen lands.** `DubRequest` gained `regen_only: list[str] | None` and a parallel `segment_ids: list[str] | None`. `dub_generate.py` honours them: if `regen_only` is set, segments whose id isn't in the allow-list skip TTS and load `DUB_DIR/{job}/seg_N.wav` from disk (resampled + padded to slot). Graceful fallback: if the cached WAV is broken, the segment becomes silence and an SSE `warning` event fires; the mix still completes. **UI**: `App.jsx.handleDubGenerate(opts)` accepts `{ regenOnly }`; DubTab shows a pink "Regen N changed" button alongside "Generate Dub" when `incrementalPlan.stale.length > 0` post-dub.
- **4.4 Rate-ratio badge.** `DubSegmentRow` now shows a compact πŸ“– `1.12Γ—` badge in the time column when `seg.rate_ratio` drifts >3% from 1.0, colour-coded (red >1.15, info-blue <0.85, muted otherwise). Tooltip surfaces `rate_error` if rate-fit bailed. `App.jsx.handleTranslateAll` propagates `rate_ratio` + `rate_error` onto each segment from the translator response.
- Build 162 ms, typecheck clean, 79 backend tests pass in 3.17 s.
### 2026-04-21 β€” Phase 4.2 + 4.4 fully wired; 4.1 signal-UX
This is the "last mile" the prior session explicitly deferred. The two defensible bets now run end-to-end.
- **4.2 Directorial AI β€” wired everywhere.**
- Schema: `DubSegment.direction` + `TranslateSegment.direction` added.
- Translator: `cinematic_refine_sync(direction=…)` + `cinematic_refine_many(directions={idβ†’text})`. Director's `translate_hint()` prepended to reflect + adapt system prompts.
- Generator (`dub_generate.py`): per-segment direction is parsed, `instruct_prompt()` appended to the segment's `instruct` string fed to TTS, `rate_bias()` multiplies `seg_speed`. So "urgent, surprised" now **simultaneously** influences the translated wording, the TTS emotive delivery, and the generation pace.
- Frontend: new `components/DirectionDialog.jsx` (live `/tools/direction` preview pane). `DubSegmentRow` menu surfaces "Set direction…" / "Edit direction…". Segments with direction show a ✨ icon + brand-tinted button background. `App.jsx` routes `openDirection` / `saveDirection` through Undo, passes direction onwards in `handleTranslateAll` and `handleDubGenerate`.
- **4.4 Speech-rate engineering β€” wired into Cinematic translate.** `App.jsx` auto-populates `slot_seconds = end - start` on every segment in the translate payload. When `quality=cinematic` and a slot is provided, the translator runs `speech_rate.adjust_for_slot` on each adapted line after reflect/adapt, returning `rate_ratio` (and `rate_error` on LLM failure). Best-effort β€” LLM outage just leaves the cinematic text unchanged.
- **4.1 Incremental re-dub β€” signal-UX.** App.jsx now snapshots segment fingerprints after every successful dub via `/tools/incremental`. `incrementalPlan` recomputes whenever `dubSegments` changes. DubTab footer surfaces a **"N segments changed since last generate"** warn badge when stale > 0, or a neutral "all up to date" when clean. The partial-regen backend path (generate filters to `stale` IDs + reassembles on top of existing tracks) is the remaining gate β€” deferred because it needs per-segment WAV retention + crossfade reassembly for a proper UX.
- Frontend build 176 ms, typecheck clean. Backend 79/1 skipped in 4.07 s β€” no new suites added; integration pieces exercise existing tests.
### 2026-04-21 β€” Phase 4 foundation + UI catch-up
Biggest single-session push so far. Three layers at once: **UI catching up to Phase 3 backends**, **four Phase 4 services with unit tests**, **two brand-new pages**.
UI catch-up (consuming what was already built but invisible):
- **`Settings β†’ Engines` tab** β€” new `EnginesTab` sub-component consumes `GET /engines`. Shows the active engine per family (TTS / ASR / LLM), lists every registered backend, badges ready vs unavailable with the actionable reason string (e.g. "install voxcpm", "set TRANSLATE_BASE_URL"). Uses the design system end-to-end.
- **`/queue` page** β€” new lazy `pages/BatchQueue.jsx` at `mode='queue'`. Three tabs (Active / Completed / Failed), polls `GET /jobs?status=…` every 3s on the Active tab. Per-job card: status badge + dot, type, project, age, duration, error, expandable `meta_json`. Consumes the Phase 2.1 job_store.
- **`/tools` page** β€” new lazy `pages/ToolsPage.jsx` at `mode='tools'`. Three interactive cards (DirectorTool, RateFitTool, ProbeTool). Uses design-system primitives only.
Phase 4 services (all tested):
- **4.1 Incremental re-dub** β€” `services/incremental.py`. SHA-1 fingerprint of generation inputs (`text Β· target_lang Β· profile_id Β· instruct Β· speed Β· direction`). Cosmetic fields (gain, selection, UI state) ignored. `plan_incremental` returns `{stale, fresh, total, fingerprints}`. 5 tests.
- **4.2 Directorial AI** β€” `services/director.py`. 5-dimension taxonomy (energy, emotion, pace, intimacy, formality) with 30+ values. Dual parser: LLM via `OpenAICompatBackend` when configured, keyword-heuristic fallback otherwise. `Direction.instruct_prompt()` flattens to a TTS instruct. `translate_hint()` produces a sentence for reflect/adapt. `rate_bias()` nudges slot-fit. 5 tests.
- **4.4 Speech-rate** β€” `services/speech_rate.py`. Per-language CPS table, tolerance window (0.92–1.08), 3-attempt LLM trim/expand loop, keeps the best-ratio candidate when exhausted. 3 tests.
- **4.6 Tools** β€” new router `backend/api/routers/tools.py` exposing `/tools/probe`, `/tools/incremental`, `/tools/direction`, `/tools/rate-fit`. New CLI `omnivoice/cli/dub.py` drives the full dub pipeline over HTTP with SSE streaming; added as `omnivoice-dub` entry point in `pyproject.toml`. 3 smoke tests added.
Bundle: main bundle 229.31 kB β†’ 230.20 kB (+0.89 kB for two new lazy pages; negligible). Build 211 ms. Total backend tests: **79 passing + 1 skipped** in 3.26 s (up from 63 before this sitting).
### 2026-04-21 β€” Phase 3 complete: pluggable engines
All four adapter interfaces shipped in one sitting, with tests + HTTP surface.
- **3.1 TTS** β€” new `backend/services/tts_backend.py`. `TTSBackend` ABC, `OmniVoiceBackend` wrapping `k2-fsa/OmniVoice` (reuses `model_manager.get_model()` so no double load), env-driven selection (`OMNIVOICE_TTS_BACKEND`, default `omnivoice`). `list_backends()` returns `[{id, display_name, available, reason}]` so the Settings-UI picker can grey out unavailable engines with actionable reasons.
- **3.2 VoxCPM2** β€” `VoxCPM2Backend` in the same file. Scaffold returns an actionable "install voxcpm + need CUDA" message until deps are present; when they are, `generate()` maps our `instruct` field onto VoxCPM2's inline `"(instruct)prompt"` syntax and supports ultimate cloning (ref_audio + ref_text). 48 kHz, 30 advertised languages.
- **3.3 ASR** β€” new `backend/services/asr_backend.py`. `MLXWhisperBackend` (default on Apple Silicon) + `PyTorchWhisperBackend` (CUDA/CPU fallback reusing the TTS model's `_asr_pipe`). Auto-detects best engine based on `torch.backends.mps`; override via `OMNIVOICE_ASR_BACKEND`. Both normalise to the `chunks` shape existing code already consumes.
- **3.4 LLM** β€” new `backend/services/llm_backend.py`. `OpenAICompatBackend` lifts the client construction out of `translator.py` so every LLM-using feature (Cinematic translate, glossary auto-extract, Phase-4 Directorial AI) goes through one code path. `OffBackend` provides an explicit no-LLM state with a clear "set TRANSLATE_BASE_URL" hint; defaults to `off` when nothing's configured.
- **Wiring** β€” new `backend/api/routers/engines.py` with `GET /engines`, `/engines/tts`, `/engines/asr`, `/engines/llm`. Wired through `backend/main.py`.
- **Tests** β€” new `tests/test_engines.py` (12 tests covering all three registries: default listings, availability messages, env overrides, sample rates per backend, auto-detect, unknown-id error, Off-backend's actionable raise). Smoke suite extended with two engine-endpoint tests. **Total backend tests: 63** (1 skipped for missing optional `openai` pkg) in 3.18 s.
- Zero breaking changes. Existing callers (`dub_generate`, `dub_translate`, `segmentation`) still talk to the direct model/pipeline β€” migration to the adapters can happen one file at a time.
### 2026-04-21 β€” Phase 2 sweep: 2.2 / 2.3 / 2.4 finish / 2.6 / 2.7
Five items closed out together in one sitting. Phase 2 progress: 5/8 β†’ **7/8**.
- **2.4 finish** β€” `ingest_pipeline` (the 260-line download β†’ extract β†’ demucs β†’ scene β†’ thumbnail async generator) moved from `dub_core.py` to `services/dub_pipeline.py`, along with `run_proc_factory` and `yt_download_sync`. `dub_core.py`: **525 lines** (down from 777, **βˆ’40.9 %** overall from original 889). Legacy aliases kept so dub_generate / dub_translate / dub_export imports keep working.
- **2.6** β€” `print()` sweep in `services/model_manager.py` β†’ `logger.info`. New `OMNIVOICE_JSON_LOGS=1` env flag installs a single-line JSON formatter on every handler (console + rotating file). Verified output: `{"t": "...", "level": "INFO", "name": "omnivoice.api", "msg": "..."}`.
- **2.7** β€” 13 new backend tests across two files. `tests/test_dub_pipeline_state.py` covers path-safety, SSE shape, process tracking, and in-memory + SQLite job round-trip. `tests/test_translator.py` covers glossary preamble, no-LLM graceful fallback, empty-literal passthrough, full 3-step chain with mocked LLM, and reflect-failure path. **Total backend tests: 50** (from 26) in 3.10 s.
- **2.3** β€” TypeScript scaffolding live. `typescript@5.9.3`, `tsconfig.json` with `allowJs: true` + `strict: true` so JSX + TSX coexist during the migration. New `bun run typecheck` script. `frontend/src/api/client.ts` migrated as the first `.ts` file; store slices authored in TS from day one. Typecheck passes clean.
- **2.2** β€” Zustand scaffolding + first three slices. `zustand@5.0.12` installed. New `frontend/src/store/` with `prefsSlice.ts` + `glossarySlice.ts`, composed via the `persist` middleware (localStorage, key `omnivoice.app`, version 1, partial persistence). Removed three ad-hoc `useState(localStorage.getItem(...))` blocks from App.jsx (`translateQuality`, `dualSubs`, `glossaryTerms`) β€” behaviour identical, now centralised.
Bundle impact: main bundle 226.56 kB β†’ 229.31 kB (+2.75 kB for zustand; negligible). Build 168 ms.
### 2026-04-21 β€” Phase 2.4 (partial): Split `dub_core.py`
**First pass of the dub-pipeline extraction.** Router no longer owns the business logic.
- New `backend/services/dub_pipeline.py` (222 lines). Public surface:
- **Pure helpers:** `compute_file_hash`, `safe_job_dir`, `sse_event`, `prep_event`.
- **Cache lookup:** `find_cached_job` β€” reuses artifacts from prior jobs with the same content hash.
- **Process lifecycle:** `register_proc` / `unregister_proc` / `kill_job_procs` / `has_active_procs` β€” tracks in-flight ffmpeg/demucs subprocesses so `/dub/abort` can tear them down.
- **Job state:** `get_job` / `put_job` / `save_job` on top of `_dub_jobs` (in-memory) and `dub_history.job_data` (persistent).
- `dub_core.py` keeps name-compat aliases (`_get_job = dub_pipeline.get_job`, etc.) so the other three routers that import from `dub_core` don't need updating yet.
- **Size:** dub_core.py **889 β†’ 777 lines** (βˆ’12.6 %). dub_pipeline.py: new, 222 lines.
- All 37 backend tests remain green in 3.09 s.
- ⏳ Deferred: the 200-line `_ingest_gen` streaming pipeline and the transcription generators stay in the router for now β€” they're tied to FastAPI's `StreamingResponse` + async-generator contract and need a dedicated pass.
### 2026-04-21 β€” Phase 2.1: Persist task queue
**Tasks survive server restart.** SSE clients can reconnect with `?after_seq=N` and catch up.
- Schema v4: two new tables in `core/db.py:_BASE_SCHEMA` β€” `jobs` (id/type/project_id/status/timestamps/error/meta) and `job_events` (id/job_id/seq/payload) with indices on status, project, and `(job_id, seq)`. `IF NOT EXISTS` β†’ no migration script needed for existing DBs.
- New `backend/core/job_store.py` (187 lines): `create`, `mark_running/done/failed/cancelled`, `append_event` (monotonic seq + per-job cap of 500), `events_since(after_seq)`, `list_jobs(status=active|...)`, `get`, `sweep_orphans_on_startup()`.
- `backend/core/tasks.py` rewritten: `TaskManager` mirrors every state transition and every `_push_event` to the store. In-memory queue stays for speed; disk is belt-and-braces. Any disk error is logged and swallowed β€” live streams never break because the store hiccups.
- `backend/main.py` lifespan: runs `sweep_orphans_on_startup()` so a crashed process's `running` jobs flip to `failed` with `"Job was interrupted by a server restart."` instead of pretending to still be alive.
- New endpoints in `dub_export.py`:
- `GET /jobs?status=active&project_id=…&limit=N`
- `GET /jobs/{id}` and `GET /jobs/{id}/events?after_seq=N`
- `GET /tasks/stream/{id}?after_seq=N` β€” replays persisted tail, then (if still live) attaches to the live listener. After a restart, clients still get the final state with no spinning.
- Tests:
- `tests/test_job_store.py` β€” 7 tests: create + lifecycle, failure carries error, monotonic seq, `events_since` filtering, per-job event cap, active filter, orphan sweep.
- `tests/test_router_smoke.py` β€” 4 new: `GET /jobs` happy path, `?status=active` filter, 404s for both job + events endpoints.
- All 37 backend tests pass in 2.83 s. Does NOT persist the work itself β€” that's Phase 4.5 (step-level resumability). This phase just ensures **honest metadata across restarts**, which unblocks the future resumability work and gets today's UI talking to a truthful `jobs` API.
### 2026-04-21 β€” Phase 2.8: Voice profile page
**A first-class home for every cloned / designed voice.** Up to now, voices were managed only via tiny sidebar cards; no way to see one in detail, edit it after creation, or see where it's been used. This adds a full page.
- Backend: `GET /profiles/{id}` (full record), `PUT /profiles/{id}` (partial update of name / instruct / language / ref_text), `GET /profiles/{id}/usage` β€” lists synth-history count + scans `studio_projects.state_json` for dub-segment usage counts.
- Frontend: new `mode: 'voice'` + `activeVoiceId` state in `App.jsx`. Back button returns to whichever mode called it.
- New lazy page `frontend/src/pages/VoiceProfile.jsx`:
- **Hero** β€” name (inline-editable), type badge (Clone / Design), lock + seed + language + created-date chips, built-in reference-audio player.
- **Details** β€” style `instruct` + language + reference transcript, all edit-in-place with Save/Cancel. Lock banner with inline Unlock.
- **Try-it** β€” type any line, generate with the current voice, auto-plays. Uses existing `/generate` endpoint.
- **Usage** β€” synth-clip count + project list (click any project to open it).
- Sidebar clone/design cards gain an "Open" action routing to the new page alongside Preview / Select / Lock / Delete.
- Pure design-system build: `Panel`, `Button`, `Input`, `Textarea`, `Field`, `Badge`, `Segmented`, `Progress` β€” zero new inline styles.
- Frontend build 168 ms. Backend tests 26 / 26 green.
### 2026-04-21 β€” Phase 1 complete: 1.4 Dual subtitle export
Closes Phase 1.
- Backend: `GET /dub/srt/{id}` and `GET /dub/vtt/{id}` both accept a `?dual=1` query. New `_pick_subtitle_text(seg, dual)` helper emits two-line stacked cues: translated on top, original italicised underneath (`<i>…</i>` β€” standard across SRT/VTT renderers). Filename gets a `_dual` suffix.
- Frontend: "Dual subtitles" checkbox in Output Options, localStorage-persisted (`omnivoice.dub.dualSubs`). SRT / VTT download buttons show a ✦ badge + switch filename + pass `?dual=1` when on.
- All backend tests stay green (26 / 26 in 2.73 s). Frontend build 154 ms.
### 2026-04-21 β€” Phase 1.3: Project-scoped term glossary
**Per-project translation consistency**, end-to-end:
- `glossary_terms` table in `core/db.py` (`id, project_id, source, target, note, auto, created_at`) with `idx_glossary_project`. Schema version bumped to 3. The `IF NOT EXISTS` in `_BASE_SCHEMA` means old DBs pick it up automatically β€” no migration script needed yet.
- `backend/api/routers/glossary.py` (new, 243 lines) β€” full CRUD (`GET/POST/PUT/DELETE /glossary/{project_id}`), bulk clear (optionally only auto-rows), and `POST /glossary/{project_id}/auto-extract` which calls the same OpenAI-compatible LLM as the translator to propose terms from the project's source segments (deduped against existing source strings, case-insensitive).
- `frontend/src/components/GlossaryPanel.jsx` + CSS β€” edit-in-place table with manual/auto badges, add-row at the bottom, `βŒ˜β†΅` / `Enter` to submit, `Esc` to cancel, "Auto" button to run LLM extraction, "Clear auto" to wipe LLM results while keeping manual entries.
- `frontend/src/api/glossary.js` β€” typed-ish client.
- `DubTab.jsx` renders the panel whenever a dub job is loaded. `App.jsx` keeps `glossaryTerms` state; `handleTranslateAll` now passes `glossary: [{source, target, note}]` into every translate call. The translator's 1.1 prompt-injection hook (already in place) finally has data to inject.
- Graceful degradation: if no LLM is configured, auto-extract returns a 503 with a copy-paste-ready env-var hint. Manual glossary still works without any LLM.
- Smoke + segmenter tests still green: 26 / 26 in 2.91 s.
### 2026-04-21 β€” Phase 1.2: NLP-aware subtitle segmentation
**Netflix-style subtitle rules** enforced on every transcription:
- New `backend/services/subtitle_segmenter.py` β€” pure-function, no ML, no network, 226 lines. Splits at (priority order): sentence terminators β†’ clause separators β†’ conjunctions β†’ greedy word-boundary fallback. Enforces `≀42 chars/line Β· ≀2 lines Β· ≀17 CPS Β· β‰₯1.2 s duration`. Folds tiny orphans into their neighbour but not across sentence terminators or speaker changes.
- Word-level timings (WhisperX / mlx-whisper output) are honoured when present: splits land on the word gap (previous word's `end`-time) so they fall on natural breaths, not mid-word.
- Wired into both transcription paths in `dub_core.py` (chunked streaming + single-shot). Runs after diarization so speaker_id propagates to both halves of every split.
- `format_subtitle_lines(text, max_chars)` bonus helper for SRT / VTT render paths that want a pre-wrapped layout.
- Tests: `tests/test_subtitle_segmenter.py` β€” 14 tests. Total backend test count now **26** (segmenter + router smoke) passing in 2.87 s. Pre-existing `test_api.py` errors are unrelated (legacy symbol references from before the router refactor).
### 2026-04-21 β€” Phase 1.1: Translate β†’ Reflect β†’ Adapt
**Cinematic translation chain** live behind a `Quality: Fast | Cinematic` toggle:
- New `backend/services/translator.py` β€” 3-step LLM chain (literal result passed in β†’ LLM reflects on tone/idiom/length/pacing β†’ LLM adapts). Bounded concurrency via `OMNIVOICE_LLM_CONCURRENCY` (default 6). Per-segment graceful fallback to literal on LLM failure.
- `TranslateRequest` gains `quality: "fast"|"cinematic"` and `glossary: [{source, target, note}]`. Glossary, when provided, is prepended to both reflect and adapt prompts.
- `POST /dub/translate` response gains `quality_used`, and per-segment `literal` + `critique` when Cinematic ran β€” set up for the Phase-1.3 three-column Translation Workbench view (see [`design/04-translation-workbench.md`](design/04-translation-workbench.md)).
- Frontend: `translateQuality` state in `App.jsx` (localStorage-persisted), new `Segmented` control in DubTab settings bar, toast on `cinematic_skipped="no-llm-configured"` pointing to the env vars.
- Works with real OpenAI, Ollama, LM Studio, Together, Anyscale β€” anything OpenAI-compatible. Env: `TRANSLATE_BASE_URL`, `TRANSLATE_API_KEY` (or `OPENAI_API_KEY`), `TRANSLATE_MODEL` (default `gpt-4o-mini`), `OMNIVOICE_LLM_TIMEOUT` (default 45 s).
- Smoke suite still green: 12 / 12 in 2.85 s.
### 2026-04-21 β€” Phase 0 complete + design-system mid-phase exit
**Phase 0 β€” Momentum** (0.1, 0.2, 0.3, 0.4, 0.5): deleted 8 orphan files, archived `legacy_gradio/`, scaffolded Alembic, rewrote 10 error messages into actionable copy, added 12-test router smoke suite (2.79 s), shipped 520 ms launch animation + `.ui-skel` utility.
**Phase 2.5 β€” Design system** (partial): shipped the complete primitive set:
- `Button Β· Panel Β· Field Β· Input Β· Textarea Β· Select Β· Dialog Β· Slider Β· Badge Β· Tabs Β· Segmented Β· Tooltip Β· Progress Β· Menu Β· Table` (14 primitives)
- Full tokens + motion layer (`ui/tokens.css`, `ui/motion.js`)
- 9 components migrated (Settings 0 inline, CompareModal, Header, Sidebar, CloneDesignTab, AudioTrimmer, DubSegmentTable, DubSegmentRow, DubTab)
- **-52 % inline styles** (388 β†’ 186)
- **-146 lines** from `index.css` (1,857 β†’ 1,711)
- **-2.7 % main bundle** despite net additions
**Phase 2.6 β€” Logging baseline** (partial): Backend runtime log file wired up, `/system/logs` and `/system/logs/tauri` endpoints corrected (bundle identifier fix + always-on `tauri-plugin-log`), Settings β†’ Logs UI wired through all three tabs with working Clear buttons.
### 2026-04-20 β€” Research + design docs
Documented `research/LEARNINGS.md` (competitor analysis: VideoLingo, pyVideoTrans, VoxCPM2), wrote the `design/` ASCII mockup set (architecture + 8 views), drafted v1 ROADMAP.md, published STRUCTURE.md after root cleanup.
### Pre-2026-04-20
MVP feature-complete: transcribe β†’ translate β†’ dub β†’ mux, voice cloning, timeline editor, diarization, YouTube ingest, selective export, project persistence, Tauri desktop shell. See git history for detail.
---
## 🧭 Design target
Intended shape captured in [`design/`](design/) β€” one ASCII mockup per view, plus a system-architecture diagram. Every phase brings the shipped product one step closer to what those views describe. When code and design diverge, one of them is wrong β€” decide which, then fix it.
---
## πŸ” Review cadence
| Frequency | Activity | Owner |
|---|---|---|
| Weekly (Friday, 1 h) | Design consistency audit | β€” |
| Bi-weekly | Status-overview update (this file) + estimate-drift review | β€” |
| End of each phase | Retrospective + exit-criteria gate + roadmap revision | β€” |
| Quarterly | Success-metric check + perf-track dedicated week | β€” |
---
*This roadmap is a living document. Every phase ends with a revision, not an append.*