Spaces:
Running
Running
Commit History
-
Add AI slide-to-transcript alignment for untimestamped images
Claude +1: Claude Opus 4.8When the images-input mode gets a transcript but the slide filenames carry no HH-MM-SS timestamps, place each slide on the timeline by content instead of failing. Locally OCR each slide (Tesseract) and ask Claude when the narration first discusses it, yielding real per-slide start times. Falls back to an even split across the transcript span when there's no API key or alignment fails. - image_input.py: add all_have_timestamps, order_by_name, build_frames_from_starts, build_frames_even_split (clamped, ordered). - llm_extract.py: add align_slides_to_transcript + SlideAlignError (text-based, so it works through Anthropic-compatible proxies). - /jobs/from-images: three-way timeline branch (order-only / timestamp-anchored / AI-aligned) run off the event loop. - Update images.html hint, README to note timestamps are now optional. - Tests for the new helpers and alignment cleanup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01…e646192 -
feat: images mode without a transcript needs no HH-MM-SS filenames
ClaudeWhen no transcript is uploaded there's no narration to align, so slide timestamps are pointless. Names now only matter when a transcript is present: * With a transcript: filenames must carry HH-MM-SS (unchanged) so narration lines up by time. * Without a transcript: build_frames_ordered_by_name sorts slides by natural filename order (slide2 before slide10) and assigns synthetic positions — no timestamp parsing, any names work. Such segments are marked "untimed": build_segments titles them "Slide N" (not "Segment N — 0:0N") and both the guide and editor templates suppress the start–end time label, which would otherwise show meaningless synthetic values. Threaded via PipelineInputs.untimed and a new Segment.untimed field. The /jobs/from-images endpoint picks the builder by whether a transcript was provided; images.html explains both cases. Tests: natural-sort ordering with no timestamps; untimed render omits meta-time and titles "Slide N"; the timed path still shows the label. …bbe4867 -
feat: make the transcript optional in images mode
ClaudeOn /images you can now omit the transcript and get a slides-only guide: each image anchored at its timestamp, no narration. Useful when you just want an accessible image deck (especially with AI scene descriptions / alt-text turned on). Wiring: * build_frames_from_images: transcript_end is now optional. With none, timestamp parsing skips the range guard and the last slide runs to its own timestamp (no known end). * PipelineInputs gains allow_empty_transcript (default False, so the video path still fails fast on a missing transcript). When True, _load_cues returns [] for a missing transcript and run_pipeline skips the zero-cues error — a slides-only guide is a valid result there. * /jobs/from-images: transcript is optional (File(None)); "no filename" is treated as no transcript. transcript_path/transcript_end stay None, allow_empty_transcript=True. Topic segmentation still requires a transcript, so it's rejected with a clear 400 when absent. * images.html: transcript fi…d9cde51
-
fix: security + bug findings from the new-feature audit
ClaudeAdversarial audit of the images-input and /edit features found four confirmed issues, all fixed: [HIGH bug] image_input.py timestamp regex fused a date/counter prefix onto the time. Delimiters were interchangeable, so "2024-01-15_00-05-30.png" parsed as 01:15:00 (the date's 01-15 fused with the time's leading 00), losing the real 00:05:30 — the slide landed ~70 min off, or the upload was rejected outright for a validly-named file. Now the two separators must be the SAME delimiter (backreference), so a mixed "-"/"_" date-time boundary can't match; matching runs inside a lookahead so overlapping candidates are all found, and the trailing in-range one wins. [HIGH security] /edit arbitrary server-file read via audio_filename path traversal. /edit renders untrusted JSON; the image guard required an inlined image_data_uri, but audio fell through to a disk read when audio_data_uri was null, and metadata_to_segment did Path(frames_dir)/audio_filename with no sanitisation — "../../../../etc/p…1ca1018 -
feat: images-input mode — build a guide from timestamped slides + a transcript
ClaudeNew input path: upload pre-exported slide images (each named with an HH-MM-SS timestamp, e.g. PNG exports pulled from a lecture at each slide change) plus one SRT/VTT, and get a study guide with no source video and no scene detection. Each image becomes a segment anchored at its timestamp; the transcript aligns to those anchors via the same downstream pipeline the video path uses. Why: deterministic (no seeking, no scene-merge threshold — sidesteps the run-to-run scene-count variance), no multi-GB video upload, and the author controls exactly which slides appear and when. Pieces: * app/pipeline/image_input.py — strict HH-MM-SS filename parser (tolerates extra metadata in the name; prefers the trailing time; uses the transcript duration to reject a date/resolution fragment mistaken for a time) and a frame builder that sorts by timestamp, windows each segment to the next slide (last → transcript end), and rejects empty sets, unparseable names, and duplicate timestamps LOUDLY …88d3d3b