Spaces:
Running
Running
Commit History
-
Add AI slide-to-transcript alignment for untimestamped images
Claude +1: Claude Opus 4.8When the images-input mode gets a transcript but the slide filenames carry no HH-MM-SS timestamps, place each slide on the timeline by content instead of failing. Locally OCR each slide (Tesseract) and ask Claude when the narration first discusses it, yielding real per-slide start times. Falls back to an even split across the transcript span when there's no API key or alignment fails. - image_input.py: add all_have_timestamps, order_by_name, build_frames_from_starts, build_frames_even_split (clamped, ordered). - llm_extract.py: add align_slides_to_transcript + SlideAlignError (text-based, so it works through Anthropic-compatible proxies). - /jobs/from-images: three-way timeline branch (order-only / timestamp-anchored / AI-aligned) run off the event loop. - Update images.html hint, README to note timestamps are now optional. - Tests for the new helpers and alignment cleanup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01…e646192 -
fix: images-mode ZIP dropped all images; retry malformed LLM JSON
ClaudeTwo issues from a real images-mode run: ZIP bundle had no images (broken guide). The /jobs/{id}/zip endpoint gated asset filenames on the video-mode scene_NNNN.jpg pattern, so images-mode uploads that keep their original names ("slide 1.jpg") were filtered out — never written to the temp dir, never bundled — leaving HTML that references static/ files that aren't there. Replace the rigid pattern with _is_safe_asset_name: a bare basename (no separators / no traversal, enforced by Path(name).name == name) with an allowed image/ audio extension. Keeps the path-traversal protection (that gate was the reason for the regex) while accepting any legitimate filename. Malformed-JSON responses dropped a slide's enrichment. The model occasionally emits invalid JSON (an unescaped quote → "Expecting ',' delimiter"), which failed the parse and left that segment blank. Now: json.loads uses strict=False (tolerates literal control chars in strings), and extract_one retries the call ONCE on a parse/vali…f09a075 -
feat: images mode without a transcript needs no HH-MM-SS filenames
ClaudeWhen no transcript is uploaded there's no narration to align, so slide timestamps are pointless. Names now only matter when a transcript is present: * With a transcript: filenames must carry HH-MM-SS (unchanged) so narration lines up by time. * Without a transcript: build_frames_ordered_by_name sorts slides by natural filename order (slide2 before slide10) and assigns synthetic positions — no timestamp parsing, any names work. Such segments are marked "untimed": build_segments titles them "Slide N" (not "Segment N — 0:0N") and both the guide and editor templates suppress the start–end time label, which would otherwise show meaningless synthetic values. Threaded via PipelineInputs.untimed and a new Segment.untimed field. The /jobs/from-images endpoint picks the builder by whether a transcript was provided; images.html explains both cases. Tests: natural-sort ordering with no timestamps; untimed render omits meta-time and titles "Slide N"; the timed path still shows the label. …bbe4867 -
feat: make the transcript optional in images mode
ClaudeOn /images you can now omit the transcript and get a slides-only guide: each image anchored at its timestamp, no narration. Useful when you just want an accessible image deck (especially with AI scene descriptions / alt-text turned on). Wiring: * build_frames_from_images: transcript_end is now optional. With none, timestamp parsing skips the range guard and the last slide runs to its own timestamp (no known end). * PipelineInputs gains allow_empty_transcript (default False, so the video path still fails fast on a missing transcript). When True, _load_cues returns [] for a missing transcript and run_pipeline skips the zero-cues error — a slides-only guide is a valid result there. * /jobs/from-images: transcript is optional (File(None)); "no filename" is treated as no transcript. transcript_path/transcript_end stay None, allow_empty_transcript=True. Topic segmentation still requires a transcript, so it's rejected with a clear 400 when absent. * images.html: transcript fi…d9cde51
-
rename: the app is now "The Describatron"
ClaudeRebrand the user-facing name from "Video Guide Maker" to "The Describatron" — and de-video the copy now that it also builds guides from slide images: * Page titles + headers across index / images / edit pages. * FastAPI app title; published-guide and editor footers. * README (HF Space card title + emoji 🎬→🤖 + heading), USER_GUIDE and DATA_HANDLING headings, and the intro prose. * Index eyebrow + sample copy: "lecture video (or slide images) and a transcript" instead of video-only. Infrastructure identifiers left untouched: the GitHub repo (VideoGuideMaker), the HF Space (Video2Guide), the internal package/ logger name (videoguidemaker), and DEPLOY.md's paths — those are real names, not branding. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJg2f32611
-
fix: close the 3 previously-unverified low-severity audit findings
ClaudeThe first bug audit had 3 findings whose verifier agents died on an API session limit, leaving them unconfirmed. Re-verified against current code — all real, all low severity, two fixed here (the third is a recommendation, below): * extract_all built its shared anthropic client OUTSIDE any try, so a missing/broken SDK (the import is lazy precisely so the app runs without the package) would abort the whole job instead of degrading to empty results — contradicting the module's "always returns, pipeline can always proceed" contract that extract_one and extract_topics honor. Guard the shared-client construction: on failure, log and return empty results with progress marked complete. * /process (the legacy non-browser endpoint) lacked the "AI features requested but no API key" guard that /jobs and /jobs/from-images have, so a script POSTing llm_* toggles with no key got a 200 with every AI feature silently dropped. Add the same fail-fast 400. Not changed (recommendation…0658e45 -
fix: security + bug findings from the new-feature audit
ClaudeAdversarial audit of the images-input and /edit features found four confirmed issues, all fixed: [HIGH bug] image_input.py timestamp regex fused a date/counter prefix onto the time. Delimiters were interchangeable, so "2024-01-15_00-05-30.png" parsed as 01:15:00 (the date's 01-15 fused with the time's leading 00), losing the real 00:05:30 — the slide landed ~70 min off, or the upload was rejected outright for a validly-named file. Now the two separators must be the SAME delimiter (backreference), so a mixed "-"/"_" date-time boundary can't match; matching runs inside a lookahead so overlapping candidates are all found, and the trailing in-range one wins. [HIGH security] /edit arbitrary server-file read via audio_filename path traversal. /edit renders untrusted JSON; the image guard required an inlined image_data_uri, but audio fell through to a disk read when audio_data_uri was null, and metadata_to_segment did Path(frames_dir)/audio_filename with no sanitisation — "../../../../etc/p…1ca1018 -
feat: self-contained edits export + /edit page to reopen a shared JSON
ClaudeMakes the editor's JSON export a true handoff artifact: one file that someone else can open and keep editing, with no accompanying images and no live job. Self-contained export (review.html): * readState now embeds each segment's current image as a data URI (and audio when inlined), so "Save edits (JSON)" carries the slides, not just references to them. The primary <img> src is already a data URI in review mode (even after alt-switching), so it's captured verbatim; a non-data src is omitted rather than saved broken. * downloadZIP strips those embedded URIs before POSTing (it sends images separately + keeps audio server-side) so the ZIP request doesn't carry the image bytes twice. * applyState restores the <img> from image_data_uri when a self-contained JSON is loaded into a live editor, so a file from another session shows the right slides. Data model (metadata.py, html_gen.py): * SegmentMetadata gains optional image_data_uri / audio_data_uri; migration + round-trip…89cae6c -
feat: images-input UI — standalone /images upload page
ClaudeAdds a self-contained upload page for the images-input mode, kept separate from the video form so the working video flow is untouched. /images page: * Multi-image file input + transcript input + title + optional API key. * AI toggles: section titles, scene description, alt-text, on-screen text, math equations. Topic segmentation is intentionally excluded — re-bucketing hand-picked slides into topics fights the point of curating exact slides (the endpoint's topic flags are always sent false from this page). * Output format picker (editor / single HTML / ZIP). * Submit posts to /jobs/from-images, subscribes to the same SSE progress stream, and on completion navigates to the editor (review) or downloads (single/zip) — reuses the existing job/events/result infrastructure end to end. * Accessible: skip link, labelled fields, aria-live progress panel, visible focus, theme-consistent styling. Clears the API key from the DOM after capture. Also: a cross-link on the main vid…31ce41a -
feat: images-input mode — build a guide from timestamped slides + a transcript
ClaudeNew input path: upload pre-exported slide images (each named with an HH-MM-SS timestamp, e.g. PNG exports pulled from a lecture at each slide change) plus one SRT/VTT, and get a study guide with no source video and no scene detection. Each image becomes a segment anchored at its timestamp; the transcript aligns to those anchors via the same downstream pipeline the video path uses. Why: deterministic (no seeking, no scene-merge threshold — sidesteps the run-to-run scene-count variance), no multi-GB video upload, and the author controls exactly which slides appear and when. Pieces: * app/pipeline/image_input.py — strict HH-MM-SS filename parser (tolerates extra metadata in the name; prefers the trailing time; uses the transcript duration to reject a date/resolution fragment mistaken for a time) and a frame builder that sorts by timestamp, windows each segment to the next slide (last → transcript end), and rejects empty sets, unparseable names, and duplicate timestamps LOUDLY …88d3d3b
-
fix: preview-cache lifecycle (recoverable retry + entry cap)
ClaudeTwo preview-cache findings from the audit: main.py — recoverable selection errors destroyed the preview (finding 14): /jobs pops the preview token, then validates the selected indices. A 4xx there (bad/negative/non-int index, empty selection) ran the cleanup branch and rmtree'd the preview dir, so the user could not correct their input and retry — they had to re-run the topic preview, which costs an LLM call. Now a 4xx re-inserts the popped preview into the cache so a corrected retry reuses it; 5xx and non-HTTP errors still drop it. main.py — preview cache had no entry cap (finding 20): A flood of preview requests could pin an unbounded amount of video data on disk for the full 30-minute TTL. Add a hard cap (24 entries); _purge_stale_previews now evicts oldest-first past the cap after its TTL sweep, removing their temp dirs. https://claude.ai/code/session_01EM8jiYxdVFy1w3LeJ9GBJgd2261c7 -
fix: lifecycle + resource-leak hardening from the audit
ClaudeRemaining confirmed findings, all lower severity: app/jobs.py — job TTL evicted live editors (finding 15): The 10-minute TTL measured from finished_at and was never refreshed by access, so a review session longer than 10 min got its job purged and the editor's lazy /alts, /ocr, /audio fetches and /result re-fetch started 404ing mid-edit. Track last_access_at (bumped on every get()) and purge on idle time = max(finished_at, last_access_at) instead. app/main.py — thread-start leak (finding 19): If the OS refused threading.Thread(...).start() after registry.create(), the job sat in the registry as "running" forever (finished_at stays None → TTL purge never reclaims it), permanently consuming a concurrency slot. Pop the job on a failed start before propagating. app/main.py — upload cap enforced too late (finding 6): The per-field streaming check in _save_upload runs only after Starlette has already spooled the full multipart body to disk. Add an early Content-Len…fbb54cc -
fix: crash + data-loss bugs from the audit (transcript parsing, download headers)
ClaudeFour confirmed crash/data-loss findings: transcript.py: * IndexError when a cue's text starts with '.' — the sentence-boundary check did prefix.rsplit(None,1)[-1] on an empty prefix ([] → [-1] raises). A leading-period cue is common in auto-transcripts. Guard the empty-prefix case. * Hourless WebVTT timestamps (mm:ss.ttt — valid per spec, common for sub-hour clips) matched none of the HH:MM:SS-only regex and produced ZERO cues — the whole transcript silently ignored. Make the hours group optional; SRT (always hours) still matches. * Trailing cue whose midpoint lands at/after the computed tail edge (video duration undershot the final cue's end) matched no bucket window and was dropped from both narration text and sliced audio. Spill it into the last segment — symmetric to the existing head spill. main.py: * A title with non-latin-1 alphanumerics (CJK, emoji, some accented letters — _safe_filename keeps them because "中".isalnum() is True) reached Content-Dispositi…4f70b55
-
editor: remove summary / key-terms / self-check (round 3)
ClaudeAccessibility team's round-three asks: * "Summary Field — redundant, remove" (now subsumed by the new detailed description paragraph from PR 2) * "Key Terms — REMOVE. This replicates the glossary." * "Self-check questions — REMOVE! This is an academic role and such questions are better placed in the platform." Scope of removal: * Editor (review.html): drops the summary <input>, the key-terms list editor block, the self-check-questions list editor block, the related CSS rules (.vgm-key-term-row, .vgm-self-check-row), the readState/applyState/syncBindings hooks, the makeKeyTermRow / makeSelfCheckRow row factories, the add-key-term / add-self-check click handlers, and the entire glossary/highlight injection inside buildFinalHTML. * Published guide (guide.html): drops the "In brief:" advance organizer, the self-check <details> block, the bottom glossary section, the highlight_terms filter usage on narration, and the build_glossary call at the top of <main>. * LLM (ll…b863381 -
editor: long-form image description field (round 2)
ClaudeAdds the "Detailed description" field the accessibility team asked for — an editable open paragraph rendered UNDER the slide image that doubles as the long-form image description. When non-empty AND included in the published guide, the <img alt> drops to "" and the image is marked role="presentation": the description carries the meaning in body text where sighted AND non-sighted readers both encounter it, so there's no need to duplicate it inside an alt attribute that only screen readers hear. The pieces: * SegmentMetadata + Segment + migration gain description / description_ reviewed fields. Round-trips through Save/Load JSON. * New LLMToggles.description toggle. _FIELD_PROMPTS adds a description prompt that follows the team's example ("A gallery of purple flower stem macro photographs is shown. These demonstrate..."), and the structured-output schema gains a matching description field that only fires when the toggle is on (no wasted output tokens otherwise). * Orchestra…cf53b59
-
Topic-preview resume: also forward summaries, not just titles
ClaudeThe topic-scene-preview endpoint computes Claude's per-topic summaries and stores them in the preview cache alongside titles, but /jobs only read out and forwarded titles. On the resume path the orchestrator's topic_summaries list stayed empty, so the "In brief:" advance-organizer line never appeared in the final guide — even though the editor showed the summary value correctly in the per-segment edit panel (because the preview's initial render came straight from the cache). Forward summaries through the same preview→/jobs→worker→run_pipeline seam as titles: - Read topic_summaries from the preview cache entry, filter by the user's curated indices, store as preselected_topic_summaries. - Thread it through _run_job_in_thread → run_pipeline as a new optional kwarg, mirroring preselected_topic_titles. - run_pipeline pre-populates its topic_summaries local from preselected_topic_summaries so the existing apply loop fires without needing a re-segmentation.0487cd2 -
Learner-focused features: summary, self-check, glossary, inline term highlighting
ClaudeTagged pre-learner-features at the commit before this so a single `git reset --hard pre-learner-features` (or non-destructive `git revert pre-learner-features..HEAD`) takes us back to known-good if anything misbehaves. Four learner-impact features ship together because they share data plumbing — same Segment/Metadata schema additions, same editor round-trip work: #2 One-sentence summary above each section. Claude's `summary` field from topic segmentation is now rendered as an "In brief:" advance organizer above the slide image. Primes the learner before they hit the detail; evidence-based for comprehension. Editable in the review screen. #4 Auto-built glossary at the end of the guide. Collects every segment's key_terms across the whole document, dedupes case- insensitively (first-occurrence wins), sorts alphabetically, and anchors each entry back to the section where the term was first introduced. #5 Inline key-term highlighting in narration. Defined terms w…d61b118 -
Second-pass audit fixes: narrow exception catch, empty-cue early error
ClaudeTwo real findings from a second-pass bug/security audit; the rest of the report was either already handled by existing code (sub-toggle sync runs on dispatched change event) or correctly identified as acceptable / non-bugs. P1 — except BaseException in the /jobs preview-cleanup block was overly broad. Narrow to Exception so KeyboardInterrupt / SystemExit still propagate without first running cleanup that could mask the signal. P2 — Empty transcript silently produced a guide with no narration under any segment. Both topic and non-topic paths now check `if not cues` immediately after _load_cues and raise PipelineError with a message naming the fix ("provide a non-empty VTT/SRT or enable auto-transcribe"). The topic-scene-preview endpoint already had its own empty-cue check; this brings the main pipeline to parity. Verified safe (no changes needed): - filename validation: endpoint regexes already cover /jobs/{id}/zip, /alts, /ocr, /audio - autoescape: Jinja2 select_autoescape([…e8ba7af -
P1/P2 bug audit fixes: index validation, cleanup race, cache purge, type whitelist
ClaudeFour real issues from a thorough audit of the recent feature work: P1 — Preview workdir leaked on validation failure. Between popping the preview cache entry in /jobs and entering the workdir try/finally, any HTTPException raised by index parsing or selection filtering would orphan the temp dir holding the user's uploaded video. Wrap the post-pop validation in its own try/except that cleans up the popped entry before re-raising. Marks the entry None so the outer finally doesn't double-delete. P1 — Out-of-range and negative selected_indices silently dropped. The previous parser used `if i.strip().isdigit()` which (a) accepts any positive integer regardless of frame count and (b) silently ignores negative numbers and any non-digit token. A user submitting "0,5,999" against a 5-frame preview got a 5-frame guide back with no error. Now we parse each token explicitly: ValueError on non-int, 400 on negative, 400 on out-of-range against the cached preview's valid index set (positional for t…1361034
-
Demote non-LLM scene picker; AI topic preview is now the primary CTA
ClaudeThe two-button row read as "pick one of two equivalent paths" but they aren't — the AI path produces meaningfully better study guides when an API key is available. Promote the topic-preview button to the primary CTA the moment a key arrives, and demote the plain "Preview scenes" button to a small text-link beneath it ("or pick slides individually without AI"). When no key is present, hide the AI button entirely and let the plain picker stand alone as the primary CTA — no dead "disabled because key missing" state. The non-LLM path stays fully functional and intentionally one click away, so users on the public Space without a key still get the demo; users worried about privacy / cost / outages keep their escape hatch; the CLI is unaffected. Backend: /jobs now rejects up-front when any AI toggle is ticked without an API key. Previously per-segment toggles silently no-op'd and topic segmentation hard-failed halfway through; now the user gets a single explicit "add a key or uncheck AI fea…908a4e1 -
Topic prompt: video-type dropdown biases boundaries to genre structure
ClaudeGeneric topic segmentation works but produces splits that ignore genre conventions — a lab walkthrough is sliced like a lecture, a tutorial like a panel. Add a per-job "video type" hint the user can pick from a small dropdown; the orchestrator prepends a paragraph of structural guidance to the topic prompt so Claude knows what shape to bias toward. Five genres modeled today (each one paragraph of prompt): - Classroom lecture: intro → background → concept(s) → examples → wrap-up - Lab walkthrough: intro → theory → materials → procedure → results - How-to tutorial: intro → setup → steps → verification → troubleshooting - Interview / Q&A: boundaries at new questions or subject pivots - Talk / presentation: hook → thesis → supporting points → closer plus "General" which leaves the prompt untouched. Unknown values fall back to general so a stale form post can't break the call. Wired through extract_topics(video_type=...), LLMToggles .topic_video_type, the orchestrator, and both …6dd9ac4 -
Add 'Preview topic scenes' button alongside the existing scene picker
ClaudeTwin of the Preview scenes flow that runs the topic LLM pipeline (scenes -> filter -> transcript -> OCR -> topic LLM -> primary picking) and renders one card per topic with title, summary, and primary-frame thumbnail. The user unchecks topics they don't want, then submits; /jobs resumes the pipeline from the curated subset and skips the topic LLM call that already ran. Saves the per-segment LLM extraction cost on topics the user wouldn't have wanted anyway. Backend - New POST /topic-scene-preview endpoint mirroring /scene-preview but additionally running OCR + topic LLM + primary-frame picking before returning topic cards. Caches the post-topic kept frames plus parallel title / summary / OCR lists alongside the same preview-cache infrastructure (is_topic_preview marker on the entry so /jobs can disambiguate). - /jobs honours topic-preview indices positionally and threads the selected topic titles through run_pipeline as preselected_topic_titles. Topic segmentation is for…edad228
-
Merge pull request #58 from JoeMartis/claude/fix-zip-path-traversal-9kA4Q
Claude/fix zip path traversal 9k a4 q25ab096 -
Add image-augmented topic segmentation as a sub-toggle
ClaudeBuilds on the transcript-only topic segmentation: when the new "Also send slide images" sub-toggle is on (only available when the parent toggle is checked), the orchestrator hands extract_topics a list of (start_seconds, image_path) tuples and the function attaches thumbnailed slides to the Claude request alongside the transcript. Slides are downscaled to 768px max-dim before base64 encoding so Anthropic's image-token cost stays bounded; if more than 60 kept scenes exist, the set is uniformly sampled (preserving the first and last frames where chapter markers usually live). Two prompt variants — the system prompt for the image branch explicitly tells Claude how to interpret slide-vs-narration signals, so it doesn't double-count layout changes within the same topic. UI: the sub-toggle is hidden until the parent is checked, and gets unchecked automatically when the parent unchecks so a hidden box can't post stale state. Tests verify the request shape (image blocks present + image-awar…9a92330 -
Add LLM topic-based segmentation as an opt-in toggle
ClaudeReplaces the scene-based one-segment-per-slide bucketing with topical sections generated by Claude from the transcript. One extra LLM call per job (transcript-only, no images) returns 3–15 sequential topics with titles and summaries; the orchestrator then re-buckets the kept scene frames so each topic-segment carries the longest-on-screen scene as its primary image and the other scenes as switchable alternates in the editor. Per the design discussion: - Opt-in via a new LLMToggles.topic_segmentation field; existing scene-based path stays the default. - Multi-slide topics keep every slide accessible (primary + cross- scene alts on top of the per-scene candidate alts). - LLM failure raises PipelineError so the job surfaces a clear error rather than silently falling back. The /alts and /ocr endpoints now also accept the scene_NNNN.jpg primary pattern because cross-scene alternates re-use those filenames.7e20c47
-
Remove X-Frame-Options and CSP headers that broke HF Spaces
ClaudeHF Spaces proxies the app through its own infrastructure; setting X-Frame-Options: SAMEORIGIN caused "refused to connect" for all users. Keep only X-Content-Type-Options: nosniff, which is safe everywhere. https://claude.ai/code/session_01EGBMur51izertyiu7msWPQ204c25c -
Fix bugs and security issues from post-a11y scan
ClaudeBugs: - _doc_styles.html: move @import to first line (was after :root, silently ignored by browsers — Merriweather/Source Sans 3 never loaded) - index.html: merge duplicate .vgm-scene-thumb rule; position:relative now lives in the main block so the ::after overlay always has a positioning context - index.html: add sandbox attribute to review iframe (allow-scripts allow-same-origin allow-downloads allow-forms) - index.html: reduce non-review format resetForRetry delay 2500→1500ms; focus is now returned sooner after download - review.html: makeKeyTermRow aria-label updates live as user types the term, matching the Jinja server-rendered row behaviour Security: - main.py: add HTTP security header middleware (X-Content-Type-Options, X-Frame-Options, Referrer-Policy, Content-Security-Policy) - llm_extract.py: scrub API key from exception messages before logging https://claude.ai/code/session_01EGBMur51izertyiu7msWPQ3964d02 -
Fix P1 bugs: cache race, blocking event loop, retry state, compat selector
Claude- _preview_cache: add threading.Lock() guarding all reads/writes/pops so concurrent preview purge + job start can't double-free a dir - /scene-preview: move PIL thumbnail loop into run_in_executor so it no longer blocks the async event loop during frame encoding - /jobs: protect cache .pop() with lock; add frames_dir.mkdir() for non-preview path (defensive, detect_and_extract creates it anyway) - /process: wrap run_pipeline in asyncio.to_thread so the backwards-compat endpoint doesn't freeze the event loop during pipeline execution - _save_upload: remove redundant f.close() inside with-block (double-close) - resetForRetry: clear preview token + restore video required attr + hide scene section, so retry after error doesn't send an expired token - review.html applyState: replace label:has() with .closest('label') for Firefox < 121 / Safari < 15.4 compatibility https://claude.ai/code/session_01EGBMur51izertyiu7msWPQ803dc05 -
Pass Advanced settings to scene preview
Claudethreshold, min_gap, max_frames, and face_threshold are now read from the form and sent with the /scene-preview request, so the thumbnail strip reflects the same detection settings that Generate guide will use. https://claude.ai/code/session_01EGBMur51izertyiu7msWPQ71306bd -
Add scene picker — preview and select scenes before guide generation
ClaudeUsers can now click "Preview scenes" after selecting a video. The server runs scene detection + instructor filtering and returns thumbnails. A scrollable strip lets users deselect unwanted scenes before submitting. The guide is then built only from the selected scenes, skipping detection and OCR for the rest. - POST /scene-preview: detects + filters scenes, returns base64 thumbnails and a short-lived preview token (30-min TTL) - POST /jobs: accepts optional preview_token + selected_indices; reuses the cached video and pre-filtered frames — no re-upload, no re-detection - orchestrator.run_pipeline(): accepts preselected_frames to skip the detect + filter stages entirely - index.html: Preview button, thumbnail strip with toggle, select all/none, count badge; video omitted from final FormData when token is present https://claude.ai/code/session_01EGBMur51izertyiu7msWPQ961990f -
Make upload size limits configurable via env vars
ClaudeMAX_VIDEO_MB=0 and MAX_TRANSCRIPT_MB=0 disable the caps for local use. HF Spaces is unaffected — defaults of 500 MB / 5 MB still apply when the env vars are absent. https://claude.ai/code/session_01EGBMur51izertyiu7msWPQa528be9
-
Add skip_inverted_ocr flag to opt out of the inverted Tesseract pass
ClaudeThe inverted-binarization pass exists to recover white-on-coloured callout text the normal pass turns into noise, but on slides with no such callouts it just halves throughput and risks producing garbled near-duplicates of body text (filtered as of the previous fix). Add a per-job toggle that disables the inverted pass entirely: - ocr_frame(skip_inverted=True) returns only normal-pass output - PipelineInputs.skip_inverted_ocr threads it through the orchestrator - /jobs and /process endpoints accept skip_inverted_ocr as a form field - CLI exposes --skip-inverted-ocr on build and metadata - Upload page advanced settings include a "Skip inverted OCR pass" toggle - Job remembers the choice so the lazy /ocr endpoint stays consistent when the reviewer picks an alt frame in the editor355b428 -
Keep review HTML re-fetchable and serialize audio_filename in readState
Claude/result/{id} freed job.result.body after the first GET, but the upload page loads review HTML into an iframe and lets the user "Open in new tab" on the same URL — the second fetch hit 410. Skip the free for review-mode results (no attachment filename) so the body stays available for the job's TTL; single/zip downloads remain one-shot. readState() in review.html serialized image_filename but not audio_filename, so the per-segment audio element's data-audio-filename never made it into the state posted to /jobs/{id}/zip. The ZIP endpoint then rendered metadata with empty audio_filename fields and silently dropped all narration audio from the bundle even though the job still held the bytes. Pull data-audio-filename off the audio element and emit it in the segment object.e845d2d -
Validate segment filenames in /jobs/{id}/zip to block path traversal
ClaudeThe endpoint trusted state.segments[*].image_filename and audio_filename when joining them to frames_dir, so a crafted name like ../../etc/hosts would resolve outside the temp dir and bundle_zip would write that file into the archive as static/<basename>. Filenames now must match the same regex patterns the pipeline produces (primary, alt, or audio) or they are dropped, keeping resolved paths under frames_dir.a037749 -
Add LLM-based OCR toggle (Claude reads on-screen text)
ClaudeTesseract struggles on slides with coloured callouts, decorative fonts, or rendered math. When the new llm_ocr_text toggle is on (and an Anthropic key is supplied), Claude transcribes the slide text directly from the frame and the result replaces Tesseract's OCR before the page renders. The OCR field is added to the existing per-frame extraction call, so segments that already pay for alt-text / key-terms / equations get OCR essentially free. Lazy alt-frame OCR (the /ocr endpoint) keeps using Tesseract — the API key isn't persisted with the job, so the lazy path can't reach Claude after the initial pipeline run.d06062f -
Add Download ZIP button to the review editor
ClaudeA new POST /jobs/{job_id}/zip endpoint takes the editor's current state plus the data URIs of every currently-picked primary frame, decodes them into a temp dir alongside the job's in-memory audio bytes, and calls the same render_from_metadata + bundle_zip path the CLI uses. The toolbar shows a "Download ZIP" button when the page was rendered with zip_url set (review mode only). Filenames are validated against the same regexes used by /alts and /audio so a crafted body can't write outside the temp dir.3df5f8f -
Second-pass audit: SSE timing, OCR memory cap, filter robustness, CLI ordering
Claude- SSE generator now resets its idle timeout on every event received rather than measuring from job.created_at, so a healthy long-running job emitting periodic progress events isn't killed at the 30-min mark. Also re-checks job.status == "running" right before calling finish_error to avoid racing a finish_success that landed during the events.get timeout. - ocr._preprocess_for_ocr caps the post-upscale max dimension at 2400 px so an unusual input frame doesn't allocate huge intermediates (the subsequent adaptiveThreshold doubles the footprint again, and we run OCR concurrently across job + lazy-alt requests). - filter_instructor_frames now skips candidate paths that don't exist on disk. Previously is_instructor_frame returned False on a missing file (cv2.imread is None), which let phantom paths slip through into the editor's frame picker where /alts/ would then 404. - cli build: dump_metadata runs BEFORE rendering, not after, so a template / disk-write failure do…9d2a95f -
Audit-driven hardening: upload caps, registry quota, SSE timeout, OCR escape
ClaudeFive small fixes from the security/bug sweep, all scoped to the existing no-auth single-user deploy model: - Cap upload size: stream UploadFile to disk in chunks and abort past 500 MB (video) / 5 MB (transcript). Replaces the unbounded shutil.copyfileobj that an anon caller could exploit to fill the Space's RAM/disk. - Cap concurrent jobs: JobRegistry.create() now purges stale entries and evicts the oldest finished entry; if the registry is still at MAX_LIVE_JOBS=8 with all of them running, raises JobQuotaExceeded which the /jobs endpoint surfaces as 429. - SSE generator no longer pings forever when a worker thread dies silently — bails out (and emits a terminal error event) after 30 min of running-but-event-less. - /ocr endpoint dedupes parallel cache writes via dict.setdefault, so two simultaneous picks of the same alt don't both run Tesseract and fight over the result. - text_format._render_block escapes the LaTeX it injects into the ocr-equation paragraph; th…7c2d809 -
Make alt-frame OCR lazy
ClaudeRunning Tesseract on every alternate frame at pipeline time tripled OCR duration on the HF free CPU tier and made the job appear stuck. Move that work to a lazy /ocr/{job_id}/{filename} endpoint and have the frame picker fetch + cache the text only when the reviewer actually picks an alt. The first click on an alt shows a brief "(loading on-screen text…)" placeholder; subsequent picks of the same alt are instant from the per-button cache (and the per-job server cache for cross-tab consistency).9a058d1 -
Default LLM enrichment toggles to off
ClaudeThe four LLM toggles (section titles, alt-text, key terms, math equations) all required an Anthropic API key and incurred per-job billing, but were checked by default in the form. Flip every default to off — in LLMToggles, both /run endpoints, and the index form — so no LLM call happens unless the user explicitly opts in.063a56d -
Default per-segment audio slicing to off
Claudeffmpeg slicing adds noticeable processing time and ~500 KB per segment for a feature most reviewers never use. Flip the default to off in the PipelineInputs dataclass, both /run endpoints, and the index form checkbox; the toggle stays so learners who do want re-listenable clips can opt in.3bed55b
-
Consolidate zip assets under static/
ClaudePreviously the zip put images in static/ but audio in a separate audio/ folder. Now everything lives in one static/ directory: scene_*.jpg frames and segment_*.mp3 clips share the namespace (different filename prefixes prevent collisions) and the bundle has a clean two-entry layout: study-guide.html static/ scene_0000.jpg scene_0001.jpg segment_0000.mp3 segment_0001.mp3 Changes: - bundle_zip writes audio_paths under static_dir_name (was audio_dir_name). - guide.html template emits {{image_dir}}/<filename> for both image and audio src, so a renamed image_dir flows through to audio too. - Orchestrator slices audio into inputs.frames_dir directly instead of a sibling audio/ dir, so freshly-built export-metadata bundles have audio in the same place metadata_to_segment expects. - main.py's worker reads audio bytes from result.audio_paths directly, no longer guessing the on-disk layout. - metadata_to_segment looks in frames_…f2b292d -
Three fixes: stale ends, /jobs cleanup window, audio toggle
Claude1. _enforce_min_gap and _cap_max_frames previously only renumbered the kept frames. Each kept frame retained its ORIGINAL end_seconds even though transcript-alignment uses the next kept frame's start as the narration boundary. So a guide could display 'Segment 1: 0:00-0:05' while its narration body actually covered 0:00 through 0:30 (when the next kept scene started). New _absorb_dropped_intervals helper extends each kept frame's end_seconds to the next kept frame's start (or to the original tail for the last). Both helpers call it after renumbering. New test pins the behavior. 2. /jobs widens the cleanup window: previously the try/except only wrapped file-save and max_frames parsing; PipelineInputs(), registry.create(), and Thread.start() ran outside it. If anything in those last three failed (dataclass validation, OS thread limits, etc.) the workdir would leak. Now everything pre-handoff is inside a try/finally with a handed_off flag; rmtree on…e7c2416 -
Sync deps, format validation, /jobs workdir cleanup, timing tests
Claude1. pyproject.toml gains 'anthropic>=0.49' so 'pip install .' brings in the same dependency that requirements.txt declares. Previously fresh installs through package metadata silently lost the LLM feature even when the API key was set. 2. /jobs now wraps the file-save + form-parse phase in try/except so if anything fails between mkdtemp and worker handoff (disk full, bad max_frames value, copyfileobj raising), the workdir is shutil.rmtree'd before the 400 propagates. Mirrors the existing try/finally pattern in /process. Test guard added. 3. Both /jobs and /process now reject unknown 'format' values with a 400 instead of treating anything-not-review-or-single as zip. The CLI already validated this; the web layer now matches. Test guard added. 4. Five new test_scenes.py invariant tests pin the timing properties that align_segments depends on after frames are dropped: * indices contiguous from 0 * start_seconds strictly increasing * end >=…06c75aa -
Per-segment narration audio (Feature 5)
Claudeffmpeg slices each kept segment's [start, end) window from the source video into a 32 kbps mono MP3 (~500 KB per 2-min segment). Below 1.5s segments are skipped — too short to be useful. Failures are non-fatal: a segment without audio just doesn't render an <audio> element. ffmpeg-not-on-PATH skips the whole stage gracefully. Plumbing: app/pipeline/audio.py — slice_segment / slice_all helpers (subprocess ffmpeg, libmp3lame, 22 kHz mono, bounded timeout, stderr captured for warnings). PipelineInputs.extract_audio (default True) gates the stage. PipelineResult.audio_paths carries Path|None per segment. Segment.audio_filename / audio_data_uri reach the templates. Render paths: - zip: audio/ folder added to the bundle; templates use 'audio/{fn}'. - single: audio_data_uri inlined as base64 alongside images. - review: lazy-fetched via /audio/{job_id}/{filename} (mirrors the alts pattern). Editor download/previ…364bfae -
Per-feature LLM toggles, section titles, redesigned upload page
Claudellm_extract: LLMToggles dataclass with alt_text/key_terms/math/section_title flags. Disabled toggles drop the field from BOTH the prompt and the Pydantic response schema (built dynamically via pydantic.create_model) so we don't pay output tokens for fields we'd discard. Disabling all of them skips the LLM call entirely. Section titles is a new field — the LLM generates an ≤8-word descriptive title per segment, which the orchestrator overrides on seg.section_title (replacing the generic 'Segment 3 — 4:25' default). main.py: four new Form() bools (llm_alt_text/key_terms/math_equations/ section_title) plumbed into PipelineInputs.llm_toggles on both /jobs and the legacy /process endpoint. index.html: full redesign. Two-column hero with a polished form card + side panel of features/tips. Drop-zone file inputs that show name, size, video duration/resolution, transcript cue count + first cue after selection. Output formats render as radio cards with description copy. AI-assist section pairs…ec46cbf -
Fix four bugs from the audit: memory, race, leak, scoring order
Claude1. /result drops result.body after first delivery (HTTP 410 on re-fetch). Previously the rendered HTML/zip was retained for the full job TTL (10 min), so a busy server could OOM. Alts stay alive on the Job object so the editor's lazy-fetch keeps working. 2. selectFrameButton sets data-image-filename + aria-pressed synchronously, before the data-URI fetch awaits. So a saveJSON() right after Load JSON reads the freshly-loaded pick instead of the previously-rendered one. As a free bonus, added a per-section pickToken so a slow earlier fetch can't overwrite a faster later pick when the user double-clicks. 3. previewURL blob now revoked on pagehide. Previously the blob leaked per session if the user previewed once and then closed the editor. 4. _save_top_candidate_frames runs gamma normalization before scoring instead of after. A dark-themed slide deck no longer gets outranked by a transient bright frame (instructor cutaway, fade flash) — the score ref…28435e9 -
Lazy-fetch alternate frame thumbnails via /alts/{job_id}/{filename}
ClaudePreviously the review HTML inlined every alternate as a base64 data URI, roughly tripling the page size. Now the editor loads only the primary inline; alternates are served on demand from a per-job in-memory cache. Server side: - Job.alt_image_bytes captures the JPEG bytes for each alternate before the worker thread tears down its workdir. - New /alts/{job_id}/{filename} endpoint returns the JPEG bytes with a strict regex on the filename pattern to block path traversal. - /result switched from pop() to get() so the job stays alive long enough for the editor to fetch alternates; the existing job-TTL purge handles eventual cleanup. Client side: - render_review takes alts_url_template; the template emits data-frame-url for lazy alts and data-frame-uri for inlined ones. - selectFrameButton became async: when picking a lazy alt it fetches the URL, converts to a data URI via FileReader, swaps the main <img src>, and caches the converted URI on the butt…597ad13 -
Embed editor in upload page; per-session API key entry
ClaudeEditor: instead of a full window.location redirect to /result/{id} once the pipeline finishes, open the review HTML in an iframe overlay on top of the upload page with a thin header bar. The form pane is preserved underneath, so 'Back to upload' instantly returns to a fresh form without a page reload. Also added an 'Open in new tab' affordance for users who want a standalone tab. Per-session API key: added an optional password-type field on the upload form. When present, it's threaded through PipelineInputs (with field repr=False so accidental log.exception(inputs) calls don't leak it) and used to construct a per-job Anthropic client. The DOM input is cleared as soon as the FormData is built, and the env-var fallback is preserved for headless CLI use. Without a key, the LLM extraction step is silently skipped and the rest of the pipeline runs as before.4af7dd3 -
Tolerate blank max_frames on /jobs and /process
ClaudeThe upload form left max_frames as a blank input by default, which posted "" and tripped FastAPI's int parser ("Input should be a valid integer, unable to parse string as an integer"). Two-pronged fix: - Front-end: strip empty optional fields (max_frames, min_gap, face_threshold, subtitle, module) from the FormData before posting to /jobs. - Server: accept max_frames as `str | None` and coerce in a small helper that treats blank/whitespace as None and surfaces a 400 for truly bad values. Regression covered: tests/test_progress.py asserts POST /jobs with max_frames="" returns 202. https://claude.ai/code/session_01G3XpEehfr9DnX9L7BahQ4Lc7cacd1